Yeah, it doesn't matter to me. I have 4's through 10's in my closet. Mostly I wear a 6, though...right now.
It fluctuates, so having a range is a good idea.
I shop in thrift stores where everything is all thrown together, and I find that manufacturer's sizes can vary
wildly. For instance, the Banana Republic and Ann Taylor sizes run large, so I can wear their 4s because they fit like 6s. Whereas The Limited and DKNY seem to be true 6s. And then there are manufacturers whose sizes run small as well, Tommy Hilfiger is one, I have to wear an 8 in Hilfiger jeans, because they seem to be made for girls without an ass, lol.
So, the moral too all that is I don't see how it's feasible to stick to one size. I HATE being in clothes that don't fit me right, especially if they are too tight.