The negative effects of the phytoestrogens on males is generally unfounded, as some have hinted here. There are no studies that show clear and conclusive data that there are negative effects to worry about.
If you are worried about estrogen levels, you should be concerned about
dairy just as much or more so than soy. The dairy industry is well known to manipulate estrogen levels and reproductive cycles to maximize the production of cow's milk. Apparently, 60 to 70% of the estrogen in our food is derived from dairy.
Actually, some studies have shown that the phytoestrogens in fact "agonize" (i.e. compete with) the more potent endogenous estrogens that act in certain cancers. This would help explain, in part, why there are fewer cases of prostate and breast cancers in Asia (and vegans in general) than in the West.
If anything, soy is a relatively healthy alternative to dairy (and meat). The main difference being that no one
suffers needlessly.