Well you just admitted it applies more to public transit because they need a police to kick them out. If I can be an asshole and no one is able to show up and stop me, I have effectively the freedom 'de facto' to be an asshole even if I do not have 'de jure' freedom to be an asshole.
Personally I'm not so sure police in NYC can kick people out for 1st amendment protected activity, which was what I specifically referenced the asshole activity being under the umbrella of. That was your assertion, that while I contested how pragmatic it might be, I never stated whether I believed it was true or not.
Drivers are given the latitude to kick passengers off public transportation for a range of reasons including violating the rules of conduct.
If I'm reading this correctly, you're talking about panhandling as an example of asshole activity that you believe is protected by the 1st amendment. Specifically, in NYC, it was ruled that panhandling is not protected speech on public transportation, see Young v New York City Transit Authority.
Of course, if the passenger refuses to leave or stop, the driver can't physically force them to and must escalate to a police force. Although, I imagine that's similar in many other countries as well.