Because all the variables that go into performance / efficiency measurement of a model (processing power, algorithm efficiency, parallelization, etc) boil down to cost per token input and token output. And the tangible cost for a datacenter is power consumed. Of course, amortized capex costs are also part of the game.
Maybe it's just because the specifics on FLOPs are more complicated, especially given how many different floating point formats are floating around in ML. Even NVIDIA has like 6 different FLOPs numbers on their GPUs nowadays.
Some of it might be market-signaling to the broader energy industry: "hey would you PLEASE build more power plants and power lines? Look at all this money we have, we will pay for it!"
It's easy to think about. Google reported a global average power consumption of 3.7GW in 2024, so you can think of this deal as representing an expansion of something like 10-15% of that 2024 baseline, if you assume 50% capacity utilization.
There's at least a decent argument to be made that the limiting factor is actually the physical silicon itself (at least at cutting-edge nodes) not really the power. This actually gives AI labs an incentive to run those specific chips somewhat cooler, because high device temperatures and high input voltages (which you need to push frequencies higher) might severely impact a modern chip's reliability over time.
Power is the limiting layer above physical chips. You can add more chips and them at lower clock or add more efficient chips later on, but you can't really change the power of a data center easily.
It will nonetheless be vastly cheaper to build a new datacenter and arrange for powering it than to fab the amount of leading-edge chips and compute systems that are going to ultimately eat that power. So the chips themselves are still the meaningful constraint.
Surely, there should be some more critical questions posed by why just buying a bunch of GPUs is a good idea? It just feels like a cheap way to show that growth is happening. It feels a bit much like FOMO. It feels like nobody with the capital is questioning whether this is actually a good idea or even a desirable way to improve AI models or even if that is money well spent. 1 GW is a lot of power. My understanding is that it is the equivalent to the instantaneous demand of a city like Seattle. This is absurd.
It feels like there is some awareness that asking for gigawatts if not terrawatts of compute probably needs more justification than has been proffered and the big banks are already trying to CYA themselves by publishing reports saying AI has not contributed meaningfully to the economy like Goldman Sachs recently did.
kinda complicated though when you consider it fully. Power consumption only measures the environmental impact really, we come up with more clever ways to use the same amount of power daily.
it's kind of like an electrical motor that exists before the strong understanding of lorentz/ohm's law. We don't really know how inefficient the thing is because we don't really know where the ceiling is aside from some loosey theoretical computational efficiency concepts that don't strongly apply to practical LLMs.
to be clear, I don't disagree that it's the limiting factor, just that 'limits' is nuanced here between effort/ability and raw power use.
"Do you realize that the human brain has been liken to an electronic brain? Someone said and I don't know whether he is right or not, but he said, if the human brain were put together on the basis of an IBM electronic brain, it would take 7 buildings the size of the Empire State Building to house it, it would take all the water of the Niagara River to cool it, and all of the power generated by the Niagara River to operate it." (Sermon by Paris Reidhead, circa 1950s.[1])
We're there on size and power.
Is there some more efficient way to do this?
Because all the variables that go into performance / efficiency measurement of a model (processing power, algorithm efficiency, parallelization, etc) boil down to cost per token input and token output. And the tangible cost for a datacenter is power consumed. Of course, amortized capex costs are also part of the game.
Maybe it's just because the specifics on FLOPs are more complicated, especially given how many different floating point formats are floating around in ML. Even NVIDIA has like 6 different FLOPs numbers on their GPUs nowadays.
Some of it might be market-signaling to the broader energy industry: "hey would you PLEASE build more power plants and power lines? Look at all this money we have, we will pay for it!"
It's more meaningful to most people than FLOPS/other measures of actual computing power.
It's easy to think about. Google reported a global average power consumption of 3.7GW in 2024, so you can think of this deal as representing an expansion of something like 10-15% of that 2024 baseline, if you assume 50% capacity utilization.
Because that’s the limiting factor
There's at least a decent argument to be made that the limiting factor is actually the physical silicon itself (at least at cutting-edge nodes) not really the power. This actually gives AI labs an incentive to run those specific chips somewhat cooler, because high device temperatures and high input voltages (which you need to push frequencies higher) might severely impact a modern chip's reliability over time.
Power is the limiting layer above physical chips. You can add more chips and them at lower clock or add more efficient chips later on, but you can't really change the power of a data center easily.
It will nonetheless be vastly cheaper to build a new datacenter and arrange for powering it than to fab the amount of leading-edge chips and compute systems that are going to ultimately eat that power. So the chips themselves are still the meaningful constraint.
I feel like that’s a bit glib?
Surely, there should be some more critical questions posed by why just buying a bunch of GPUs is a good idea? It just feels like a cheap way to show that growth is happening. It feels a bit much like FOMO. It feels like nobody with the capital is questioning whether this is actually a good idea or even a desirable way to improve AI models or even if that is money well spent. 1 GW is a lot of power. My understanding is that it is the equivalent to the instantaneous demand of a city like Seattle. This is absurd.
It feels like there is some awareness that asking for gigawatts if not terrawatts of compute probably needs more justification than has been proffered and the big banks are already trying to CYA themselves by publishing reports saying AI has not contributed meaningfully to the economy like Goldman Sachs recently did.
kinda complicated though when you consider it fully. Power consumption only measures the environmental impact really, we come up with more clever ways to use the same amount of power daily.
it's kind of like an electrical motor that exists before the strong understanding of lorentz/ohm's law. We don't really know how inefficient the thing is because we don't really know where the ceiling is aside from some loosey theoretical computational efficiency concepts that don't strongly apply to practical LLMs.
to be clear, I don't disagree that it's the limiting factor, just that 'limits' is nuanced here between effort/ability and raw power use.
Somehow we must be doing this wrong.
"Do you realize that the human brain has been liken to an electronic brain? Someone said and I don't know whether he is right or not, but he said, if the human brain were put together on the basis of an IBM electronic brain, it would take 7 buildings the size of the Empire State Building to house it, it would take all the water of the Niagara River to cool it, and all of the power generated by the Niagara River to operate it." (Sermon by Paris Reidhead, circa 1950s.[1])
We're there on size and power. Is there some more efficient way to do this?
[1] https://www.sermonindex.net/speakers/paris-reidhead/the-trag...
pretty sure evolution spent more time and energy getting there then we ultimately will
I'd imagine one day there will be a limiting factor of cash to burn as well.
We're getting close. The first big AI bankruptcy can't be far off.
Lol well OAI is falling apart at the seams.
Simo takes a medical leave. And there appears to be friction between the CEO and CFO.
Big Gov will bail out the big guys if/when necessary
[dead]