LLMs are deterministic[1], but the only way to determine the output is to empirically run them. With compilers, both the implementor and a power user understand the specific code transformations they are capable of, so you can predict their output with good accuracy. I.e. LLMs are probably chaotic systems.

edit: there might be a future where we develop robopsychology enough to understand LLM more than black boxes, we we are not there yet.

[1] Aside from injected randomness and parallel scheduling artifacts.