Nothing against what I said in there. Distillation is far from enough to get to the frontier. Its at best a ramp (the most efficient one used by everyone ) that shortcut and saves millions of rl runs before a model moves.
Nothing against what I said in there. Distillation is far from enough to get to the frontier. Its at best a ramp (the most efficient one used by everyone ) that shortcut and saves millions of rl runs before a model moves.
"No evidence" -> Large lab saying that they use distillation when training their models
And probably saves a lot of energy which would be good for the planet
You said there isn't evidence of distillation. Yes there is