I'm not sure you know what 'privilege' means. No fault to you, based on your self-described history, but it's far more than warm meals and beds to sleep in.
Make it through this, if you can. It's an excerpt based on the timeless essay by Peggy McIntosh..
Alas, a blog The Male Privilege Checklist
This is simply male privilege. There are a number of privileges associated with being white, too.
Think about it sometime. Privilege is the counter-point to 'bootstrappy' conservatism, but it's unfortunately largely ignored by those who'd benefit most from understanding it.
McIntosh's "Invisible Knapsack" if you're interested:
http://www.amptoons.com/blog/files/mcintosh.html
All of these things contribute to socioeconomic status, whether we like it or not. It's par for the course to excerpt individual points and say "ooh, but *I* didn't have *that* privilege" to discredit it, but understand that privilege acts on a society-wide level, and your individual experience does not a case against systemic inequality make.