> A tired programmer usually knows he's tired
6 months ago everybody talked about how AI was not able to plan their work. They could only do minor changes.
The introspective property of knowing when you are "tired" and acting appropriately will likely be trivially observed in 6 months.
human fatigue is detected by sensory experience and there's no sensory experience for llms. so it can just guess.
operators will never expose this guess to the user (who wants to pay for a tool that complains and refuses work?) so they probably will use it to load balance differently. and if they are out of resources then they can't load balance so they'll just hope you won't notice, so we're back to where we started. maybe it's already roughly how it works now...
what's most likely in 6 months is a big correction in pricing, once they get people hooked up on this enough. maybe there will be a model that manages to detect when it's stupid but it won't be available outside FAANG and DOD
Ahh sorry, I see that you explicitly wrote that deteriorated performance was due to throttling.
Naturally, LLMs do not get fatigue. As such they also don't need to detect it. That is correct. But LLMs can see if their performance have fatigue-like deterioration and correct for it.
That case is even more trivial and has nothing fundamentally to do with LLMs. Capacity is added everyday.
I would not base my long term projects on what happens in the market based on the fact that a single provider needs to throttle to satisfy demand now.
Your assumption that the current pricing is not sustainable might be fair. Personally I believe the opposite, and I havde not seen any indication that inference should not continue to decline in price.
In particular, LLMs appear to be hyper commoditisable. So if Anthropic or OpenAI is doing pricing shenanigans, people will likely move on.
> Naturally, LLMs do not get fatigue
they are not human so human sensations don't apply. but I heard when you run out of tokens it's also kind of like fatigue, just less predictable