I have no moral problem with prostitution, as long as the prostitute entered the profession of her own will, is of age, and is making her own decisions regarding STD's, drug use, abuse, etc.
However, honestly I would think twice (if not several more times) about being with a man who had been to a prostitute, perhaps because it confers a certain level of desperation/lack of self control in his character, and the increased risk of STD's. Perhaps I am stereotyping, but in choosing someone to be intimate with, I think I'm within my rights to discriminate in that manner.
If the man told me about previous prostitutory (?) experience right up front, before we ever had sex, then maybe we could work through it. But if I found out afterwards (i.e. he was not fully disclosing his sexual history), I'd be hard-pressed to not kick him in the balls and to the curb shortly afterwards.
But, this is all hypothetical, since the situation does not apply to me.

I'd much rather be with a mid-20's virgin

(as I was) or anyone with "normal" experience, than be with someone who was so desperate to lose his virginity that he paid someone to get rid of it for him. Instant "loser" status, right there.