It might be fun to swap houses with a relative. I have many that live nearby, and we seem to stay in groups somewhat even when moving out of state. I think that I would feel more comfortable knowing who was living at my house, especially if there are people that I know who are nearby to help me see the sights. So I don't think swapping with strangers would be something I would do.
I would, however, make an exception for Bill Gates.
edit: oh, and I saw the movie "The Holiday." I liked the movie, but disliked the acting of Cameron Diaz.