The compute power is also ridiculous.
Some of the reason for the high density is that you need devices physically close to each other to share such bandwidth. It’s not because we’re limited by the physical building space, because we can construct buildings all day long. Sending bits around at ultra high speed is hard and you need to keep all of the devices physically close to avoid having your interconnect costs explode.
Interestingly the realm in which I have domain experience has similar constraints, but based primarily on physical transport latency and less on bandwidth. There has been a move in some spaces towards hyper-dense deployments, but it’s a very small amount of the total compute capacity due to other limitations.
Still, the world I’m used to operating in is typically 5-10 kVA/rack.