With respect to consumption, it’s pretty efficient vs older traditional servers, though I know workloads like that aren’t completely fungible. Nonetheless it bears keeping in mind that a single GB200 NVL72 rack provides 1.4 ExaFLOPS of AI compute (at FP4 precision, ideal circumstances, but this is envelope math all around). So it’s power efficient, for what it is.
Oh, I have no doubt it is functionally efficient. I'm just amazed given the system deployments I've been party to, and the tiny amount of per rack energy usage comparatively speaking given the functionality of those systems.
Like, what in the good god damn are we using all this energy for?
You left out overthrowing governments with customized targeted propaganda, jamming citizen discussion with noise, artificially creating and nourishing contrarian cells in democratic societies. The machines will now be programming people.
In theory the water stays clean and can be reused. But I assume these cheapskates will go for evaporative cooling everytime? Then yeah, we need laws against that.
Some of the reason for the high density is that you need devices physically close to each other to share such bandwidth. It’s not because we’re limited by the physical building space, because we can construct buildings all day long. Sending bits around at ultra high speed is hard and you need to keep all of the devices physically close to avoid having your interconnect costs explode.
Interestingly the realm in which I have domain experience has similar constraints, but based primarily on physical transport latency and less on bandwidth. There has been a move in some spaces towards hyper-dense deployments, but it’s a very small amount of the total compute capacity due to other limitations.
Still, the world I’m used to operating in is typically 5-10 kVA/rack.
With respect to consumption, it’s pretty efficient vs older traditional servers, though I know workloads like that aren’t completely fungible. Nonetheless it bears keeping in mind that a single GB200 NVL72 rack provides 1.4 ExaFLOPS of AI compute (at FP4 precision, ideal circumstances, but this is envelope math all around). So it’s power efficient, for what it is.
Oh, I have no doubt it is functionally efficient. I'm just amazed given the system deployments I've been party to, and the tiny amount of per rack energy usage comparatively speaking given the functionality of those systems.
Like, what in the good god damn are we using all this energy for?
> Like, what in the good god damn are we using all this energy for?
Bad AI porn, terrible AI music, AI scams and completely devastating the labor market.
And based on the recent Anthropic/Pentagon rift... I guess also creating autonomous kill-bots and doing mass surveillance.
Just a bunch of super cool stuff.
You left out overthrowing governments with customized targeted propaganda, jamming citizen discussion with noise, artificially creating and nourishing contrarian cells in democratic societies. The machines will now be programming people.
Since you and me and everyone else will foot the elecricy bill. Energy consumption or efficiency is not a concern.
It's the water use that concerns me
In theory the water stays clean and can be reused. But I assume these cheapskates will go for evaporative cooling everytime? Then yeah, we need laws against that.
The compute power is also ridiculous.
Some of the reason for the high density is that you need devices physically close to each other to share such bandwidth. It’s not because we’re limited by the physical building space, because we can construct buildings all day long. Sending bits around at ultra high speed is hard and you need to keep all of the devices physically close to avoid having your interconnect costs explode.
Interestingly the realm in which I have domain experience has similar constraints, but based primarily on physical transport latency and less on bandwidth. There has been a move in some spaces towards hyper-dense deployments, but it’s a very small amount of the total compute capacity due to other limitations.
Still, the world I’m used to operating in is typically 5-10 kVA/rack.
There are box trucks with less power consumption.