Madrid is a beautiful city to wander around in...I'd advise against a car, as the traffic in Madrid is insane. But, if you like extreme sports, go for it.
If you're not up for museums, I'd say just wander around. No better way to explore a city, IMO.
Toledo is an interesting old city just about 45 minutes via bus from Madrid, so if you get bored, that might be an interesting place to hit up.