Or however deterministic inference is supposed to happen. I don’t know LLMs.
https://152334h.github.io/blog/non-determinism-in-gpt-4/
https://152334h.github.io/blog/non-determinism-in-gpt-4/