I use
http://www.best-machine.com at the recommendation of one other TFP user. I've been relatively pleased with them, and considering the low monthly price, I can't complain.
They've answered all of my technical questions within 45minutes in every occasion.
I've also used
http://www.powweb.com without any serious issues. I've heard good things about
http://www.lunarhost.com too, but in the end, best-machine.com was less expensive and gave me everything I needed.
But, if you're just interested in plugging around, then a free host might be more in order. Maybe
http://50megs.com/ would be a good fit? It all depends on how much you want to learn, and how much you are willing to pay.
PS: Powweb used to allow you to mail in a check if you didn't have access to a credit card. You had to pay quarterly, which was the drawback to this setup, but you're only talking about $24...