It all depends...
A girl I know used to call me a dork, but it was in a very friendly manner, and usually was said after one of my lame jokes/comments.
I didn't mind this at all, and knew it was just a way of her to say "something" in response to a joke. It wasn't intended to be hurtful and I didn't take it as so.
Even when intended to be hurtful, the person calling you a nerd is at least acknowledging that they think you are smarter than they are.
