These aren’t raw base models they are the result of a ton of RLHF and various adjustments.
Bitter lesson wildly overstated in this context.
These aren’t raw base models they are the result of a ton of RLHF and various adjustments.
Bitter lesson wildly overstated in this context.
More RLHF is in fact scaling.
Yes, but not in the “dump another chunk of all written language in the bucket and stir”-sense which is what bitter lesson became synonymous with.
That may not be the intent of the original article, but over the past few years that’s what the phrase turned into.
GPT-6 is supposed to be using a much larger base model that just finished pretraining so the "dump another chunk of all written language" approach is still going strong.
Modern pretraining also consists of expensive human-led specialized task creation and grading loops. Synthetic generation and distillation from previous models is another input for training. I wonder how much new text contributes beyond keeping knowledge up-to-date.
rlhf = reinforcement learning from human feedback
(had to look it up)
The scaling with reasoning models is more and more with things like verifiable rewards (coding and math), in line with bitter lesson and also Sutton invented lots of modern RL.