AI crap has already been crammed into everything for months now, and nobody like it nor wants it. There is no proof that AI will continue to improve and no certainty that it will become a disadvantage not to use them. In fact, we are seeing the improvements slow down and it looks like the model will plateau sooner rather then later.
> There is no proof that AI will continue to improve and no certainty that it will become a disadvantage not to use them. In fact, we are seeing the improvements slow down and it looks like the model will plateau sooner rather then later.
While I expect the improvements to slow down and stop, due to the money running out, there's definitely evidence that the models can keep improving until that point.
"Sooner or later", given the trend lines, is sill enough to do to SWEng what Wikipedia did to Encyclopædia Britannica: Still exists, but utterly changed.
> While I expect the improvements to slow down and stop, due to the money running out
This will certainly happen with the models that use weird proprietary licenses, which people only contribute to if they're being paid, but open ones can continue beyond that point.
The hyperscalers are buying enough compute and energy to distort the market, enough money thrown at them to distort the US economy.
Open models, even if 100% of internet users joined an AI@HOME kind of project, don't have enough resources to do that.
You are right, machine learning models usually improve with more data and more parameters. Open model will never have enough resources to reach a compatible quality.
However, this technology is not magic, it is still just statistics, and during inference it looks very much like your run of the mill curve fitting (just in a billion parameter space). An inherent problem in regression analysis is that at some point you have too many parameters, and you are actually fitting the random errors in your sample (called overfitting). I think this puts an upper limit on the capabilities of LLMs, just like it does for the rest of our known tools of statistics.
There is a way to prevent that though, you can reduce the number of parameters and train a specialized model. I would actually argue that this is the future of AI algorithms and LLMs are kind of a dead end with usefulness limited to entertainment (as a very expensive toy). And that current applications of LLMs will be replaced by specialized models with fewer parameters, and hence much much much cheaper to train and run. Specialized models predate LLMs and we have a good idea of how an open source model fares in that market.
And it turns out, open source specialized models have proven them selves quite nicely actually. In go we have KataGo which is one of the best models on the market. Similarly in chess we have Stockfish.