>>Intuitively, it seems to me that there must necessarily be some kind of upper limit
I mentioned this in other thread(https://news.ycombinator.com/item?id=48514481), we are at runaway intelligence already.
Mostly because we are looping AI to fix problems, and then the same data is used to improve AI. There is no upper limit to this.
Taken to its logical conclusion, this process needs a hardware scale that might even look laughably huge at this point. Its fairly obvious space is going to play a big role in the coming times.
I could be wrong, and I humbly accept it when Im proven wrong. But it does feel like a lot of people in top places know we are going to need all the energy and resources space has to offer to run this runaway intelligence.
> we are at runaway intelligence already. Mostly because we are looping AI to fix problems, and then the same data is used to improve AI. There is no upper limit to this.
I'm an AI booster, but this really doesn't follow. There's absolutely no guarantee that the marginal improvement from this continues to hold.
Its not marginal improvement.
Validated data coming out of LLM(Itself generated from a recursive/loop process, used to incrementally arrive at solutions) being using to improve the very LLM is a very powerful loop. And there is no real upper limit to this, at least not in the near future.
Like most exponential processes, the start is slow, but it gets fast very rapidly.
This is also what Anthropic has said, the ones training these models today(i.e 2025 - 2027) will be impossible to catch up with, let alone beat.
So we are at a kind of runaway AI already today.
Economics meaning of marginal, not the qualitative one.
> there is no real upper limit to this
This is what you need to cite for your argument to hold together.