I think it's becoming clear that these mega AI corps are juggling with their models at inference time to produce unrealistically good results. By that it seems that they're just cranking up the compute beyond reasonable levels in order to gain PR points against each other.
The fact is most ordinary mortals never get access to a fraction of that kind of power, which explains the commonly reported issues with AI models failing to complete even rudimentary tasks. It's now turned into a whole marketing circus (maybe to justify these ludicrous billion-dollar valuations?).
Models drop in price x10 each year. Us, common folk, getting access to these kinds of models is just a matter of time.
Is that true though? Having to pay some $200 a month for a max account of whatever kind doesn't seem to be cheaper to me at all?
$200/month for an LLM with the capability to fully automate my job is extremely cheap. Of course, even with a high thinking budget we don't have that yet, but if we see it at any cost in 2026, I'll be expecting to be forced into retirement by 2030.
When I say 10 times cheaper, I mean when comparing models of the same capabilities. The kind of performance you get now for a 200$ subscription, a year ago probably would have costed 2000$.
I don’t believe that current models are 1000x better than the initial ChatGPT release. What metric are you using?
You don't? Now I use Gemini to code and optimize CUDA kernels. When I first used GPT3 in the OpenAI playground I was extremely impressed when I managed to get it to output a hello world program in C.
I understand what you're saying. However I'm not sure it's that germane when we're talking about whether or not the current $200 subscription fee is actually delivering value for money, or whether AI giants are manipulating performance to gain marketing points.
I assume the original reply was addressing the “never” in this specific point:
“The fact is most ordinary mortals never get access to a fraction of that kind of power”
Since previous generations of models get aggressively retired the cost reduction essentially never gets passed down to the customer.
A certain amount of input and output tokens doesn't cost 10x less than before.
Ok but if they can pump those compute and get science/math advancements it's worth something even if the costs are very high
ICPC problems are about as far from scientific advancements as a spelling bee is from Shakespeare.
(I’m a former ICPC competitor)
The bleeding edge behind closed doors token burning monsters of 2023 are bad compared to the free LLMs we have now.
I believe it was Sundar in an interview with Lex who said that the reason they haven't developed another Ultra model is because by the time it is ready to launch, the flash and pro versions will have already made it redundant.
But then why does every new model release work great for a few weeks, then suddenly performance plummets? It's mysterious?
"It's now turned into a whole marketing circus (maybe to justify these ludicrous billion-dollar valuations?)."
Yes theres an entire ecosystem being built up around language models that has to stay afloat for another 5 years at least, to hope for a significant breakthrough.
I think part of it depends on whether you see AI progress as research or product.
Very good point.