If this were true, then a 10fps movie clip would be indistinguishable from a 24fps or a 60fps one. I have written several years ago about how optimizing my shell prompt from 50ms to 5ms was definitely a noticeable impact on how snappy the shell felt: https://xyrillian.de/thoughts/posts/latency-matters.html
The context was about UI interactions, not at all about movie clip which is a totally different thing.
I've occasionally spent time doing and even fighting for latency optimisations that supposedly don't matter in the great scheme of things, but that resulted in customers leaving positive feedback about how the product is noticeably more responsive and/or feels more polished than the competition in those specific areas. It can definitely make a difference.