The "normal" sperm concentration for a man is between 80 and 120 million per millilitre, and the average of men who ate on average a portion of soy-based food every other day was 41 million fewer.
statistical proof that being a vegan makes you less of a man
I know vegans that don't eat soy.
Soy has its own set of problems, much like meat/dairy, so some people avoid that, too.