I uh guess I’m not a normal person then
Working with soft synths, the difference between 65ms, to 15ms latency, 8ms latency, and 2ms latency - time from pressing the key to speakers emitting the sound - is agonizingly noticeable.
The numbers I’m quoting are ones I remember from various gear and upgrades over the years. It’s crazy to think about the levels of latency I was stuck with when I was a poor college kid. These days I wouldn’t settle for more than 10ms latency, and I don’t have to, thank the maker.
I would say when working with synths, the difference between 15ms and 2ms is just swing, it is noticeable but it doesn't feel wrong, just makes things more interesting.
When a drummer plays drums all the hits are off-timing 10-40ms, and it is still considered natural-sounding swing, if it is agonisingly noticeable for you – you have a quite robotic sense of rhythm, it's very subjective after all.