The actual constraint is how long people are willing to wait for results.
If the results are expected to be really good, people will wait a seriously long time.
That’s why engineers move on to the next feature as soon as the thing is working - people simply don’t care if it could be faster, as long as it’s not too slow.
It doesn’t matter what’s technically possible- in fact, a computer that works too fast might be viewed as suspicious. Taking a while to give a result is a kind of proof of work.
> people simply don't care
I don't think that's right, even for laypeople. It's just that the pain of things that take 5 seconds when they could take 50 ms is subtle and can be discounted or ignored until you are doing a hundred things in a row that take 5 seconds instead of 50 ms. And if you don't know that it should be doable in 50 ms then you don't necessarily know you should be complaining about that pain.
It's also that the people who pay the price for slowness aren't the people who can fix it. Optimizing a common function in popular code might collectively save centuries of time, but unless that converts to making more money for your employer, they probably don't want you to do it. https://www.folklore.org/Saving_Lives.html
> It doesn’t matter what’s technically possible- in fact, a computer that works too fast might be viewed as suspicious. Taking a while to give a result is a kind of proof of work.
In recent times I found myself falling for this preconception when a LLM starts to spit text just a couple of seconds after a complex request.
https://thedailywtf.com/articles/The-Slow-Down-Loop