I think we're going to suddenly see them greatly scale back training and try to sell inference-only, but they all know when they do that someone can jump up and outstrip them.
I think we're going to suddenly see them greatly scale back training and try to sell inference-only, but they all know when they do that someone can jump up and outstrip them.
But only as long as training actually improves models significantly. As soon as those improvements stay below a certain threshold, the better move is to invest your R&D money into other things like harnesses or new tricks one can play with existing models and the immense cost of training is just not worth it to be 0.5% ahead.
I'm absolutely certain that we will reach that point, just not when. Could come sooner than we think though.
Don't get me wrong I still want the models intelligence to improve, but for all practical purposes we are already there this is why many people are already moving to cheaper/open source. There is still a case though for the 1% of queries that demand SOTA
At that point they would lose all advantage stemming from their ability to boil the ocean though.
> I think we're going to suddenly see them greatly scale back training and try to sell inference-only
Remember a few weeks ago when all the AI labs said "we need to slow down, to uh, prevent destroying the world"?
Ding!
In this version of conspiracy theory, all the labs secretly understood that training wasn't economically feasible anymore so they all jointly made it look like they were stopping for safety reasons.
Is there no end to this kind of lazy conspiracy theory
You don't need to communicate to coordinate. All these companies have the same business model, if it was financially advantageous for one of them to push that narrative, then it's financially advantageous for all of them.
Or all the big ones are content with the current status quo and willing to compete amongst each other and want to shut the door to upstarts.
You see it time and time again.
So the more likely answer is they are on the verge of making super intelligence that could kill us all?