My eye glazed over a bit during the opening paragraphs, but once you get to the meat of the article about how OpenAI's own researchers are using their tools it gets a lot more interesting.
I noted that they use the acronym RSI (for Recursive Self-Improvement) without defining it. I think that's a little out of touch - I don't think RSI is a well-known acronym outside of OpenAI's bubble yet.
I actually think a goal of the current crop of OpenAI posts is expressely to reset the spectrum by normalizing the concept of RSI as something normal and safe to pursue.
The message is running through all of them. It's a mix of marketing and pacifying the intelligentia.
It's timed this way because the term is not yet well known outside the safety debate circles, so they get to frame it now.
Instead of something to fear, it will be accepted as the next step. In approximately two days the groupie crowd will write LinkedIn posts about how Sam is winning because they have the better RSI, and this will become the new standard wisdom.
In a month an AI expert will try to sell you a webinar on how to enable "RSI" in your org and your inbox will ask you if your team is doing the "RSI" yet.
> It's timed this way because the term is not yet well known
The basic concept has been here since llama3, in the open models. Likely earlier in closed labs. You use the previous gen models to curate and prepare data for the next gen. Now with the added benefit of actual arch/algo improvements (also public since gemini 2.5 gaining 1% efficiency on training next gen). This has been known for at least 2 years, in the open.
The basic concept has been there probably for 100s of years - you can go to the stuff the thinkers Mary Shelley was inspired by with Frankenstein, and you'll find similar ideas about feedback loops in science development.
I'm talking about current-era messaging and how it's being introduced to the mass public now, though.
Yep, it’s exactly this
I've been RSI'ing for 6 months.
You’re not the target audience. OpenAI communication is for the broader public, decision makers, journalists, their cultists, etc
Indeed, many programmers might pattern match to repetitive stress injury and think of their brushes with carpal tunnel syndrome. :)
Yeah, I kept looking for the first place it was defined in the article and... nothing
They must have picked that habit up from Claude...
Same. Defining acronyms should become a habit when writing.
Re-become a habit. It has long been standard good writing to always define an acronym on first use.
RSI started when humans discovered tool use.
I mean one could argue that RSI always begins in any physical environment.
The book "What is intelligence?" by Blaise Aguera is great
Are you sure that was not iterative improvement?
Iteration and recursion are famously equivalent
Everyone in AI used to know this.
But you get more funding when you call it Recursive Self Improvement. Even better if you call it RSI so it doesn't evoke pesky skynet scenarios outside of AI safety circles.
I've had (computer-related) rsi off and on for the last few years too, do not recommend
Both agents and hunans get rsi, it’s just moving them in opposite directions.
Using tools to build tools is recursive.
It's not recursive when it's done iteratively, or are you imagining GPT Astra designing GPT Galactia, which starts designing GPT Oh-My-God-ica before it has finished being created itself?
That sounds more iterative than recursive.
Recursion requires feeding the output back into the input, so creating version 4 requires results from version 3. You cannot recur in parallel.
Iteration does not. You can iterate in parallel.
You can search in parallel, but a depth N search can only become a depth N+1 search after the depth N is done (i.e. sequentially).
In any case the name RSI has stuck - the idea doesn't change or make any more sense by giving it a different name.
Because "depth" is recursive.
You can search twice without waiting for the results of your first search: iteration.
You can't if the thing you need to search for is the results of your first search: recursion.
Here's the concept.
Version 1 -> Version 2 -> Version 3 -> ...
You can call it krispy kreme donuts if you want to.
The “recursive” part comes from the fact that you have an AI which was developed by an AI (that was developed by an AI (that was developed by an AI (…)))
Sounds like "recursively" walking to the grocery store by putting one foot in front of the other (that put itself in front of the other (that put itself in front of the other (...)))
I think it's only recursive from the perspective of the humans, i.e. they design Astra, which itself as part of its deployment designs Galactica, etc.
So humans develop things one after the other, but when the thing itself starts developing new things, those are happening 'recursively' in its scope.
What is the difference between?
RSI is a fetishistic term among the singularity crowd, who imagine AI "recursively" improving itself in some exponential fashion until there is a bright flash of white light and it reveals itself in the form of god. Or something like that.
I don't know why whoever coined the term chose "recursive" rather than "iterative" - just sounds more likely to lead to infinite regress I suppose.
This notion of recursive/iterative self-improvement, whereby generation #1 AI improves itself to create generation #2, then generation #2 further improves itself to create generation #3, etc, seems to conflict with the reality that what we have with LLMs is models whose performance/capability is defined by data, not code, so the most you can do is have your LLM design synthetic data, or just do Karpathy-style "auto research" where all you are doing is using the LLM to automate your experiments.
At the end of the day, each experiment, designed by a person and/or LLM, then needs to compete with all your other ideas for compute to be tested at scale, and no amount of recursion or self-improvement will materialize an infinite amount of compute out of thin air, so your recursively synthetic-data gobbling LLM will continue to improve at the same pace it ever did.
I felt like the scaling laws were magical thinking, but apparently they work. However I still do not understand why we should expect exponential improvements due to this automated process. My intuition is that the first iteration of it should result in a noticeable capability increase (though I think these labs were already using a lot of AI to orchestrate training the current model anyway), and then the second iteration of it should be nearly identical in capability to the first, unless more data is involved, more compute is involved, or the model is bigger.
Fundamentally the current language-model approach is lacking in any general reasoning ability, so they are trying to mitigate this by using synthetic data and reinforcement learning to bake specific reasoning chains into the model, one domain at a time ... coding, math, hacking, three.js competence ...
The trouble with this is that there is little generalization in the utility of these baked-in reasoning chains from one domain to the next, so in the end this is not dissimilar to the CYC project's decades long attempt to encode all of human knowledge into a giant expert system... the hope is that if you make your collection of jagged narrow intelligences sufficiently large then it will look more like general intelligence, not a bed of nails.
I would assume that the gains from this type of test-time compute (and synthetic RLVR dataset) scaling will level out just the same as gains from human training set scaling eventually levelled out, and basically for the same reason - because you are tapping into a finite data pool, whether language itself, or reasoning steps isolated from that language, so at some point the incremental gains become increasingly small (10->20% is a doubling, 90->95% is just a ~5% gain).
It's not clear where all the different AI companies are currently focusing - on some of these narrow verticals, or on growing the forest of narrow intelligences. OpenAI's chief scientist, Jakub Pachocki, said that their current focus is on RSI(!) - improving the model in ways that will help them iterate faster in order to have a "fire meets fire" tool than can combat enemy AIs. It's not clear what this really means - what skill set makes an LLM more helpful in the process of building LLMs, but it seems to basically be process automation.
You can get exponential growth from completely ordinary feedback loops. You start with some amount of stuff, you do a series of steps and you end up with more of the same stuff you started with. As you keep going through the loop, the stuff you have grows exponentially. That's for example how exponential economic growth works.
Of course data, compute and model size are not held constant. You start with some money and use it to acquire researchers, data and compute, and have the researchers produce a big model and you use that model to get more money, and you use the additional money for more researchers, more data, and more compute to produce a bigger model. This is what has propelled exponential AI progress so far.
Recursive self-improvement is invoked to predict superexponential growth. The idea is that instead of only using the model to make more money, you add it to the researchers to speed up the loop, so not only is the money growing with every iteration, the iteration time also gets shorter, producing growth that is faster than exponential.
The problem with this simplistic prediction is that it assumes additive and multiplicative relationships of the form money = (researchers + AI)×compute_spend, but if doing more research paid off so reliably, you could also just hire more researchers, abstractly money = research_spend×compute_spend and with a balanced allocation of research and compute, you would get a money-squaring machine even without using AI for AI research.
And the reason this doesn't work in reality is that there are diminishing returns everywhere. You can also see this in the OpenAI post, where they write 7 times as much code to run 1.6 times as many experiments, and those additional experiments probably only result in minor improvements to model quality.
AI can compress AI nearly losslessly.
“Recursive” is a reasonable term because the generation N AIs will train the Generation N+1 AIs. The term “iterative” doesn’t reflect this nuance as well IMO.
Recursion reduces each step toward a base case: each step is defined in terms of previous/simpler steps, not more advanced ones. The "recursive" in "recursive self improvement" has things precisely backward. Iteration correctly describes a process where each step is the starting point of its successive step, so it should be "iterative self improvement" but I guess that didn't sound as cool.
I think you’re conflating the direction of definition with the direction of evaluation.
Compare the similarity of:
With: The latter is a classic example of recursion. So why isn’t the former?Edit: formatting
[dead]
It's not a nuance, it's a sequence.
Yes the exponential self improvement folks have never heard of an eigenvalue I guess. You can loop forever using output as input but at some point the result will stop changing (depending on the function)
I think the limit of what can be achieved with RL and synthetic data generation is better simply described as a leveling off of gains as you extract all the intelligence and knowledge from the original human training data.
Of course things will change at some point in the future as we go beyond LLMs, to build creative intelligence not just imitative/predictive intelligence, but right now these companies are stuck in this loop of building synthetic data and RLVR training from that, which means they are essentially building the "generative closure" of the original human training data - trying to squeeze all the juice out of it.
To go beyond this they need to add creativity of some sort to generate data that is not ultimately based on the original human training data. They could try something like brute force search (cf agent swarms/graphs), but this is just a more thorough way of exploring the search space defined by the training data - it may find you the "move 37" or low-hanging mathematical proof, but as Demis Hassabis has said, the goal of AGI is not to find move 37 but rather to create something capable of inventing as compelling a game as Go in the first place.
The name you are looking for is "fixed points", not "eingevalues".
that's not really how eigenvalues work... they specifically also model the case where the result keeps changing exponentially.
The claim is that the RSI operation is just finding a fixed point of improvement,
RSI(LLM) = RSI(LLM) -- for an optimal LLM* which is a fixed point of RSI
As for eigenvalues/vectors, they're fixed points of (1/val)A or A*val
Eigenvectors represent fixed directions, not fixed magnitudes. From Wikipedia:
> More precisely, an eigenvector v of a linear transformation T is scaled by a constant factor lambda when the linear transformation is applied to it: Tv = lambda v .
In other words, repeated multiplication of an eigenvector by a matrix can still create exponential growth.
>AI "recursively" improving itself in some exponential fashion until there is a bright flash of white light
Sounds like repetitive stress to me.
>loop forever using output as input but at some point the result will stop changing
Running in place will eventually wear you out too. Plus with some things it can be difficult to know for sure if that's where you are at the time.
Even worse may be if you were almost running in place, it could be orders of magnitude more difficult to discern, especially if the scale was massive to an unprecedented degree.
What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation?
How do you think why there's this fad of producing general purpose humanoid robots?
> How do you think why there's this fad of producing general purpose humanoid robots?
For doing physical work?
So a swarm of robots builds the shell of your fab overnight, and then what? Where is the EUV machine coming from?
So far the most we're seen TeslaBot do is serve drinks via tele-operation, and I don't think it's exactly built for construction site work.
For example, TSMC uses behavioral cloning to scale up human-bottlenecked parts of the manufacturing process to meet the growing demand, while automated research laboratories do thousands experiments in parallel to find better manufacturing processes.
The production bottleneck in a fab isn't the human workers - the process is mostly automated. The bottleneck more derives from how many wafers per hour you can process, which comes down to the etching process and EUV throughput.
ASMLs EUV machines are literally the most complex machine that mankind has ever built, which is why no other country, including the US, has yet been able to duplicate it. It's not just the machine itself, but a global supply chain of irreplaceable components such as focusing mirrors made by Zeiss to an incomprehensible level of accuracy - differences in surface height no more than the size of a hydrogen atom (or if you scaled the mirror up to the size of the country of Germany, then surface differences in height of 0.1mm).
Robots are useful to automate things, but they are zero help when trying to build tech like this that you are incapable of building in the first place.
The US has fallen way behind in manufacturing expertise, and no swarm of robots is going to help.
You've missed a part where it's TSMC that does behavioral cloning (to build more EUV machines). The full vertical integration is a bit farther down the line.
Etching a model's weights on silicon is another way to utilize non-top-notch tech-processes, while maintaining or improving performance. (and it suits robotics well)
TSMC doesn't know how to build EUV machines - they are stuck buying them from ASML like everyone else.
Putting a model's weights in read-only memory close to the processor is certainly a way to increase token/sec generation speed, but of course does nothing to increase intelligence. Robots aren't going to help though - semiconductor manufacturing is semiconductor manufacturing regardless of whether you are etching GPUs or memory onto your wafers.
Ah, sorry, it's ASML expertise that needs to be cloned and scaled up. I don't see how it changes things, though.
Robots don't need that much intelligence. High-speed joint control, "hand-eye coordination", the higher level tasks can be delegated to external models. Distillation already works quite well for isolating the required functionality.
I'm pretty sure ASML, and their supply chain, do have plans to increase production, as do the chip fabs - they all see the demand, while at the same time being leary of boom and bust which is the historical reality of the chip business.
But, the production expansion rate of none of these companies is being limited by lack of trained personnel, and if it were it would surely be faster to hire/train more humans since robots are still very far from human dexterity, not to mention intelligence.
Robots and AI are tools of automation, a way to replace humans with machines, but not all the problems in the world are bottle-necked by lack of humans, or the cost of humans.
Investing in training a person gets you one trained person. Investing in training an ML system gets you a cloneable ML system that can be scaled on demand much faster. ROI might change quickly.
Sure, but we're simply not at the point, maybe never will be, where lack of employees is the bottleneck to chip production. A fab takes billions of dollars and multiple years to construct - there are many constraints.
> What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation?
Money, regulations, EUV machine lead-times, global helium supply, reality ...
It's funny that we've got the Dwarkesh contingent saying that GPUs will become infinitely expensive, and now another contingent saying that they will become infinitely abundant.
Even if compute were free, and/or the AI was so smart that it picked the right experiments to run every time ("make no mistakes"), you still have to actually train the model, which takes months, and if model Ver. N+1 depends on model Ver. N, then it's iterative regardless of how much compute you have.
Who's saying that compute will become infinitely abundant? "Singularity" is just a way of saying that known models begin to give absurd predictions. Anyway, intelligence is a way of overcoming obstacles. 10 million tonnes of helium is a nice head start and retraining models from scratch is not guaranteed to last forever.
AFAIK the notion of a/the technological "singularity" is a point in time where technology is building upon itself (RSI!) so fast, at an ever increasing pace, that the speed of change effectively becomes infinite and incomprehensible to humans.
The word "singularity" is presumably coming from math or space, like a black hole singularity where matter becomes infinitely dense and the known laws of physics break down.
> 10 million tonnes of helium is a nice head start
Yeah, but then you need to refine it to 99.9999% purity, to be able to use it.