I definitely agree that it comes down more to the person than the vehicle they happen to be driving/riding.
I've noticed that driving on the shoulder/white line can result in overthinking and even sudden moves when I know a car is behind me, and thus I don't take issue at all with a concerned beep that lets me know a car is coming up on me. When riding, I'm generally staying in a very tight area, and I'm quite good at not weaving all over the place, but other drivers don't know this; they are likely seeing me on the road for the first time. If I do swerve a bit out, that could spook them and cause them to overreact, even though I'm pretty good at correcting myself and staying straight.
I've also found that looking back over the shoulder to check for traffic often results in my body following, resulting in a slight swerve, which can be troubling if there is in fact a car coming up behind me. One of the guys at the bike shop I go to has mirrors that attach to to the bottom of his handlebar grips (Google search result example:
http://www.aspirevelotech.com/Listings/Mirror.htm) and he's said these mirrors have saved his life a few times on the road.
There is a multi-path use I was on yesterday in the earlier portion of my ride, and those can be tricky, as you often have families on bikes, people walking, and runners. I'm very respectful of others when I ride, but even then, a kid riding with his family can swerve suddenly or cause a potential incident, so it's best to be careful in general.
Country roads are great to ride on, and while they do have less traffic, they also tend to have less shoulder, more blind turns, less visibility and thus just as much risk of accident. I guess the key is to just be as safe and considerate as possible and hope for the best. They're called accidents for a reason, but it's certainly possible to reduce the chances of one occurring.
I think it would be nice if people had a more open mind and put themselves in the shoes of others while on the road. When driving in an urban environment, I've had cars follow behind me on stretches of road without any parked cars, only to have them pass me when I get to an area with cars parked along the curb. I can't help but imagine myself driving into an recently opened car door, and this train of thought is generally followed by images of how badly I'd be hurt when this in fact happened. One of the images that is stuck in my head, likely until the day I die, is of a UCI Pro Tour cyclist riding down a cobblestone hill in the Gent-Wevelgem race only to fall directly on his face and be mangled, badly. If nothing else, it motivates me to drive as safely as possible so that I minimize the odds of myself being involved in such an accident; though I don't see myself driving on cobblestones any time soon.