I do not really know why anyone would wear a symbolic accessory if they did not identify with the symbolism, but I do not think it is wrong to wear such a thing. I amo not one to wear something that puts me in a select group of people, but w/e floats your boat.
I just realized I am wearing a surfing shirt... I didn't even know what was on my shirt today until I read Wnker85's post.