Yes, but for one thing, floating-point math is non-associative so even if you use the exact same random seed, you can get different results on different implementation kernels.
For another, the dependence on a random seed is just one example of how you can get different behavior. You could just as well get different results due to e.g. a tiny difference in prompt wording or punctuation or whitespace.
If a model passes some test on 70% of random seeds, and fails on the other 30%, then knowing the rough probability of success is much more useful than knowing one random sample of the outcome.
And empirically this kind of probabilistic success is quite common, because many LLM benchmarks report "pass@k" success rates that are much higher than the "pass@1" success rate on a single attempt.
I don’t understand what “floating-point math is non-associative” has to do with the random selection of possible result token set after the soft max operation. The weights of the model are set and don’t change during the many matrix operations(which is deterministic regardless of the floating point encoding scheme).