I was about to cite a source, and then I realize that it was just worded in a way that made it sound like the majority of people do not have a hallucinogenic response.
In any case, I've spoken to roughly 15 people about this, and of those only 4 or 5 were. It isn't statistically significant, but 75% of statistics are made up on the spot anyways.