> I don’t think it means the model has internalised the meaning of it (as witness when you talk to an llm how often it forgets what you recently told it was important etc).
Humans forget stuff all the time anyway. Would you give them the same diagnosis?
Btw, what you describe about 'the most probably next token' would be true for a model that only went through pre-training where they only train on exactly that task.
But there's a lot of re-inforcement learning afterwards.
> But there's a lot of re-inforcement learning afterwards.
That just shifts the distribution of tokens produced. Ultimately they are still just next token predictors.
Like, even "reasoning" models basically work by generating more tokens at inference time, and using them to shift the distribution towards more useful outcomes (in some cases).
They are next token producers. I would only call it a predictor, if it's trained to predict tokens (ie just after pretraining).
Just like humans produce one word after another when they talk, but they don't generally try to imitate other humans.
Don’t people pick up language, vocabulary and dialect from those around them? Perhaps it’s subconscious but humans are imitating other humans all the time?
It's a mix.
Yes, you imitate how others speak, but when you are trying to solve a problem, you don't try to predict how others would complete a text.
(Well, unless you follow 'what token would Jesus pick?' / 'what would Jesus do'.)
What does it matter what humans do? We're talking about LLMs, running known+vastly less complicated algorithms on known+vastly less complicated hardware.