> Like, what does a 28MB binary get you?

For one thing, on beefy-enough recent CPUs, you could keep the weights hot in the L2 cache of a single CPU core. (Which is clearly not the use-case, but might be interesting to those looking for extreme TPS numbers. Or perhaps for efficient training!)