Errr.. guilty
It's just an instinctive thing for me anyway. You can't really tell what's going on in a "sizing it up" situation with clothes on anyway. Tends to be misleading. I was just trying to think of a time when I all out stared, and I couldn't really. (edit: yeah I can. Just recently in fact. I don't think he noticed, and if he did I don't think he would have minded much!)
I think it's just human nature. We're still animals after all.