> I'm well aware that there are parts of the world where intervening will get you into trouble (and have been in situations where I have held back)
You clearly have more street smarts than the average person. The average person doesn't know when to hold back. They will say and do dumb things, and they will be killed.
There are examples right there in your comment.
> the worst of it in reality was criminals cussing at me as they retreated
You allowed them to leave even though they were insulting you, thereby avoiding violence.
Plenty of people out there who would do the opposite of what you did: they'd go out of their way to insult and humiliate the criminals as they were leaving. "Teach them a lesson", as they say. This can easily escalate the situation into lethal force.
If you insult a man in front of his peers, tell him he's a pussy right in front of his friends, you almost leave him no choice but to come back and escalate just to prove you wrong. It seems obvious but there's plenty of people out there who have died over disrespect.
> something as simple as yelling "I already called the cops"
You were smart enough to back up your threat before confronting the criminals.
Plenty of people out there who threaten the criminal with the 911 call itself. "Stop or I'll call the cops". Not only is it a direct challenge to the criminal, it also provides them with the solution to their problem: kill the guy and he won't call the cops.
It all seems obvious when we're academically discussing this stuff here but in a rapidly escalating, potentially violent situation where emotions and adrenaline are running high, people will do and say all kinds of stupid shit. And they are going to die for it.