The problem is when, as jwoody alluded to, when prostitution mixes with the drug trade. Crack whores and other hangers-on that live in and around open-air drug markets and will perform sexual favors for drugs and/or money to buy drugs won't go away or put on a pretty face, if you make prositution legal or not.
Upper-class and middle-class prositution (escort services, etc.) will probably be improved, but the typical "guy drives into a shady neighborhood, waves over a woman, gets a BJ, hands her money, and drives off" will never really go away as long as there are people desperate for drugs. Furthermore, then you have the associated crimes of what happens when a crack-whore gets assaulted by a john, or a john gets robbed by the crack-whore, and goes to report it.
There's a lot of different crime going on all in one location and it tends to be pretty intertwined, so it's hard to pick a particular violation and say "okay, we'll make this legal and a lot of our problems will vanish."
