There are indeed several options. In any case, you will a USB interface. I am using
(1) a Tascam US 200, which has two input connectors for XLR and audio jack plugs; thus, you can simultaneously record via mic and pickup which gives it a nice stereo sound. It’s a terrific device.
(2) a Behringer UCG102 Guitar Link USB Audio Interface, which has one input connector for an audio jack plug. It’s small and light-weight and thus suitable for traveling or onstage recording.
As for software, Audicity is free and absolutely sufficient. If you want to add effects, bass lines etc., try Garage Band. The Tascam should come with a copy of Cubase, another popular choice.