(1) LLMs collapse and start outputting garbage after a number of tokens if you do not sample and just pick the "best token" each time. This is a consequence of how they are trained.
(1) LLMs collapse and start outputting garbage after a number of tokens if you do not sample and just pick the "best token" each time. This is a consequence of how they are trained.
> LLMs collapse and start outputting garbage after a number of tokens if you do not sample and just pick the "best token" each time.
LLMs are likely to get stuck even with sampling if asked to generate tokens on their own long enough, though sampling does tend to stretch out the time before that happens (as do other techniques that don't involve sampling, like applying repetition penalties directly to token logits). But LLMs generally aren't left to infinitely extend their own output, and the length response typically needed in the use case is much shorter than the would result in collapse given the kinds of inputs expected in that use case, the existence of the theoretical eventuality may not really matter.
Citation needed
You know you can just try it and see on any inference system thst has this knob, right?
Related: if you don't have a limit on sampling (top-K or top-P), eventually you'll hit one of the really unlikely tokens by chance and then the model will switch to Japanese because the most likely completion after a random Japanese character in the middle of an English sentence is more Japanese writing, not a reversal back to English.
That could be how it works, but in practice it takes into account all previous tokens when producing the next-token distribution to sample from. So a switch back is more likely than your explanation supposes.