Seeing the dramatic differences in scores just going from high to xhigh is just another demonstration of the bitter lesson: Just keep scaling search and learning. We are probably going to need a lot more GPUs.
Seeing the dramatic differences in scores just going from high to xhigh is just another demonstration of the bitter lesson: Just keep scaling search and learning. We are probably going to need a lot more GPUs.
These aren’t raw base models they are the result of a ton of RLHF and various adjustments.
Bitter lesson wildly overstated in this context.
More RLHF is in fact scaling.
Yes, but not in the “dump another chunk of all written language in the bucket and stir”-sense which is what bitter lesson became synonymous with.
That may not be the intent of the original article, but over the past few years that’s what the phrase turned into.
GPT-6 is supposed to be using a much larger base model that just finished pretraining so the "dump another chunk of all written language" approach is still going strong.
Modern pretraining also consists of expensive human-led specialized task creation and grading loops. Synthetic generation and distillation from previous models is another input for training. I wonder how much new text contributes beyond keeping knowledge up-to-date.
rlhf = reinforcement learning from human feedback
(had to look it up)
The scaling with reasoning models is more and more with things like verifiable rewards (coding and math), in line with bitter lesson and also Sutton invented lots of modern RL.
While I think this is true, remember as we get more efficient we just decide to scale even bigger. So more GPUs, and more efficient.
I agree with the sibling comment, effiency is probably the more important component at this point. We are hitting not just a practical engineering roadblock for scaling with current technology, I think we have definitely hit a financial and logistical roadblock for up scaling with the number of GPUs (on an immediate basis)
Not always, in some cases, changing to a higher reasoning makes the AI doubt itself too much, and skip over the correct answer by overcomplicating the problem and polluting the context.
It would be nice to see on which categories of problems the extra thinking makes it better and on which it makes it worse.
There goes my plan to buy a PC for the next decade
The whole of knowledge work is being automated. We've barely begun to see the GPU build out. This is just the start.
I'd imagine they're going to 10x this, maybe 100x this.
Kind of refreshing though that the "throw more processing at it" scaling we saw in the 90s has returned in a different way. For a while we were really bottlenecked in our advances by relatively low levels of parallelism (most software used by your average user doesn't scale cleanly with more than a few threads).
I mean, theoretically you can solve every finitary problem with a brute force solution...
Richard Sutton specifically states that the search has to be smart. We know that the brain uses recurrent connections and is shallow. I think a lot more money has to go into architecture. Feed Forward transformers can only scale so far
Or a lot better efficiency.
> Dramatic difference
Isn't this just the difference between getting 0 right and getting 1 right?
And a lot more electricity to power them.
And dozens of data centers in every state so tokens are dirt cheap.
This isn’t really how it works anymore. Agents rely heavily on tool use and the agentic harness to perform tasks. Pre-training is no longer very effective.
I thought models werent allowed tools on arc-agi?
> We are probably going to need a lot more GPUs.
Or a breakthrough in algorithms etc.
The human brain, heck all bio brains, are proof that you don't need a lot of power or size for intelligence.
The human brain has 80 billion neurons and a 100 trillion synapses. I think you're underselling the processing power of that warm chunk of meat.
The real message of the last 15 years has actually been the opposite: if you throw enough processing power at it, intelligence emerges.
Moreover we've known for quite a while now that glial cells also participate in cognition and moderate learning (e.g.: [1]). When you take those connections into account the numbers get really staggering. 85 billion glial cells with trillions of protein channels facilitating communication between the glial syncytium [2].
[1] https://www.sciencedirect.com/science/article/pii/S193459091... [2] https://pmc.ncbi.nlm.nih.gov/articles/PMC5063692/
The real question is not how many "weights" the human brain has (neurons+synapses may or may not translate into "weights", and brain might be also inefficient for what it is), but rather how much evolutionary and social "compute" was necessary to pack everything into that capacity.
20 watts for inference AND training!
For intelligence, I expect the next breakthrough to be colocation of memory and compute in the same chip. And we'll need much more of this memory, probably a few petabytes.