Is there a better term for self-improvement where the model is actually improving _itself_ (e.g., adjusting its own weights in real-time upon learning new, verifiable information)? I’ve heard online learning.
The way “recursive self-improvement” is being used recently seems to be intentional marketing. A model helping code/train/test the next generation of its model family isn’t technically “self”-improvement.
Online learning is the term, yes.
> The way “recursive self-improvement” is being used recently seems to be intentional marketing. A model helping code/train/test the next generation of its model family isn’t technically “self”-improvement.
The term RSI is ~20 years old at this point, and the concept is even older. If an AI can't continuously modify itself and instead has to build a successor that builds a successor, that changes some specifics (for example, the alignment problem becomes more relevant - how can the AI ensure that its successor shares its goals?) but it's still a situation that may produce exponential growth, and hence most RSI-related considerations still apply.
(In fact, as a toy example, note that it's in theory possible to get exponential growth in capabilities even if you hold the software fixed, if e.g. you have an AI that can design and build better hardware to run itself faster on, which lets it design even better hardware even faster.)
Although I don't disagree with your general assessment, I do think it's a bit misplaced.
The model's parameters are larger than the weights of the neural network used for in the model. Hell, the model could be written as just a huge Python file full of if-else statements, and that code would still be correctly identified as the model's parameters. So it's a bit obtuse/arbitrary to decide that the term "recursive self-improvement" should be limited to just updating the model's neural net's weights.
Doesn't mean there shouldn't be more precise terminology, mind you. But I believe the way everyone has been using the term "recursive self-improvement" is perfectly accurate given how general that term is.
> The model's parameters are larger than the weights of the neural network used for in the model.
What do you mean by this ? The model is the neural network itself and the parameters of the model are the neural network weights (+ non-trainable parameters).
Your take works in a vacuum, but it lacks the nuance of understand what a model is and what constitutes a parameter in that greater context.
>The model is the neural network itself
No, and especially not with these LLMs. If someone handed you the "raw" neural network of a modern LLM, you couldn't just run forward passes through it and expect it to behave the same way a proper agent behaves. This should be obvious, but that's an important distinction. Everything else you need to make that neural network behave like an agent are apart of the overall model that we're talking about here. And any parameter of that "everything else" are effectively parameters of the agent itself.
Think about a model as a function: you give it some input and it produces some output. A simple model could be something literally like a linear function, f(x) = mx + b. Here, you could say f has two parameters: m and b. That all makes sense and "scales" up to neural networks where there's a lot more parameters, but it's basically just a bunch of these functions where the weights are like m and b in the linear function.
But that's not really the full picture for an LLM, right? For example, if you talk about an agent, there inputs you give it would be your prompt and the outputs it produces is whatever text it generates. But what about its system prompt? If that effectively remains fixed, then that's not really an input of the 'function' that is an agent, right? But it does influence the output of the agent. So, from a certain perspective, the system prompt is actually a parameter of the agent. Tweaking that parameter can cause it to produce different outputs for the same input. More importantly, tweaking that parameter can cause it to produce better outputs than it would otherwise. So if you have one agent test another agent by tuning its system prompt such that the output of the second agent is better than it was before, you just had one agent optimize the parameters of another agent. And that's just the system prompt. Think about the overall harness (where the actual codebase can be considered a parameter) or how multiple agents interact with each other and the parameters involved with those kinds of processes.
I'm willing to bet you don't buy that explanation, but I can assure you that in machine learning there is a lot of consideration for the parameters which come outside the actual weights of the model, and tuning those parameters can be just as important as tuning the weights of a neural network. I'll say that understanding and accepting this was one of the most important realizations I had while studying ML.
Long term memory?