Hmm, that's no good.
Unless you have a physical problem I'm not aware of, and I apologize if you do, working out should not hurt. To what I believe as the truth, if it does hurt, that means you are over doing it. I went through it myself when I first started going to the gym. I wanted to lift at the same level that my roommate lifted, even though he had been working out for over three years and I was just starting.
As far as there being weird people... I'm not quite sure I follow you on that. Weird on what way? I mean, there are weird people everywhere you go..
It does take some time, but it is time well spent in the long run, IMO. Since you belong to a 24 hour gym, why not wake up an extra hour and a half to head over there (1.5 hours to give time for driving however far it may be and coming back and showering)? Or perhaps at the end of your day you can go spend some time there.
Whatever your decision, I wish you the best of luck and good health! (that goes for everyone else too
)