Laptops/desktops are cheaper per flop than any datacenter hardware by a good order of magnitude.
The problem is that expectations rise in datacenters, hardware/power/security/availability guarantees cost real money. Then the operator providing these guarantees expects some margin.
You can see this most clearly with "developer desktops", a gcp instance costs about 10x a hetzner instance which costs between 5 and 10x the same hardware sitting in the back of an office somewhere. While all of these premiums matter for 24/7 systems under active development, they don't really matter for ephemeral small scale workloads.
Doesn’t it flip around for small scale? Paying 100x the cost for something, all in, it’s cheaper to rent for small workloads like 10m/day.
At 10x you have to be at hours per day and 5x you’re at 4h.
Actually they wouldnt spend the money if it were cheaper.
HBM has way higher bandwidth and its not all about flops.
Also the FP4 flops (inference) are so mind bogglingly high on these things.
Lastly what you fail to consider is the chip to chip bandwidth which is critical.
the people running these know that networking is just as critical.
all reduce etc.
they wouldnt pay if they could get something better value.