> When we ask for a fiction or novelty, the AI can give it to us because its processing is in part stochastic. Every decision can go multiple ways and will go different ways and produce a different trajectory every time. The trajectory can be random—and thus novel—or it can be based on the training data—and thus “good” because the training data is good, sourced from people or reality. Thus, the trajectory is either novel or good—based on randomness or based on data—but never both at the same time.
This doesn't seem true? You can be both random and based on training data.
I think they meant more "it can be extrapolated or interpolated" or "it can be high variance and 'creative' or it can be low variance and 'reliable/correct/likely'". If you want to see something new, the model will need to step off the manifold. But the manifold is where you've learned the "correct" solutions live.
Only to a limited extent.