How do you know those people aren't staring at you because you started staring at them in an unpleasant way in response to them randomly glancing a look vaguely in your direction?

I don't stare at people. They don't stare at me either. Problem solved?