I wear in the direction that supplies the most shade to the direction from which the sunlight is coming (especially if out for a long time and have stupidly left home without a larger "all around" hat). Saving my pale skin from the sun is much more important than how I look.
We pale skinned computer nerds have to protect our look, you know