Gay here, but I've only experienced a concerning conversation once, and that was a longer trip where sometimes you find out too much. I took an exit ramp away from that topic of conversation and it was fine. Otherwise everyone has been decent to downright pleasant.
I'd feel like I'm losing something by giving up that human interaction, such that it is.