“No one can make you feel inferior without your consent” ~ Eleanor Roosevelt
Yes, that's kind of a good and empowering way of looking at things. Unfortunately, what happens when the creation of inferiority was imposed before you were in any kind of position to give meaningful consent?
This is where the harm and damage comes from: early childhood, when you're perceived as different and the only thing that the rubes can latch onto is the charge of male homosexuality? And for the time, place and context, that was about as toboo as one could possibly get.
So, kinda like yeah, the toxic impact hits you before you've had the chance to build a psychic firewall to protect yourself from it.