> Expert execution is deterministic (pure matmul).
Isn't that only true in theory but wrong in practice due to floating points?
> Expert execution is deterministic (pure matmul).
Isn't that only true in theory but wrong in practice due to floating points?
There are fixed-point models too which can be deterministic. But for floating point you are very instruction-set dependent, never mind floating point operations not being commutative.
The harder case isn't same-hardware determinism; it's that lumabri wants CPU and GPU peers in the same swarm. Accumulation order differs across hardware, not just across runs.
[dead]