leaving the tag on her hat was country and grand ole opery star minnie pearl's trademark.

Heh, here in SF some would say a guy wearing his hat backwards means he's a bottom. :p

Really though, you have to take into account the entire persona, the whole outfit in addition to the circumstances. Are you out at a nice restaurant wearing jeans, tennis shoes and a ball cap backward? Are you in shorts and a tee at the beach? Totally different reaction in my book.