Every time I read one of these articles the main issue I have is that it doesn't take into account the huge shortages of compute that are going on all the time. Anthropic and Google especially have been incredibly unreliable, struggling to keep up with demand.
Each of the main providers could easily use 10x the compute tomorrow (albeit arguably inefficiently) by using more thinking for certain tasks.
Now - does that scale to the 10s of GWs of deals OpenAI is doing? Probably not right now, but the bigger issue as the article does point out in fairness is the huge backlog of power availability worldwide.
Finally, AI adoption outside of software engineering is incredibly limited at work. This is going to rapidly change. Even the Excel agent Microsoft has recently launched has the potential to result in hundred fold increases in token consumption per user. I'm also suspect of the AI sell through rate being an indicator that it's not popular for Microsoft. The later versions of M365 copilot (or whatever it is called today) are wildly better than the original ones.
It all sort of reminds me of Apple's goal of getting 1% in cell phone market share, which seemed laughably ambitious at one point - a total stretch goal. Now they are up to 20% and smartphone penetration as a whole is probably close to 90% globally of those that have a phone.
One potential wild card though for the whole market is someone figuring out a very efficient ASIC for inference (maybe with 1.58bit). GPUs are mostly overkill for inference and I would not be surprised if 10-100x efficiency gains could be had on very specialised chips.
the huge demand exists right now because the cost of a token is near zero. and companies have figured out one weird hack to gaining value in the stock market, which is to brag about how many tokens are being crammed into all manner of places that they may or may not belong.
customer value must eventually flow out of those datacenters in the opposite direction to the the energy and capex that are flowing in
do people actually want all this AI? I see studio ghibli portraits, huge amounts of internet spam, workslop... where is the value?
> Each of the main providers could easily use 10x the compute tomorrow (albeit arguably inefficiently) by using more thinking for certain tasks.
That's true for everyone with regard to any resource.
The question is whether the 10x increase in resources results in 10x or more increase in profit.
If it doesn't then it doesn't make sense to pay for the extra resources. For AI right now, the constraint is profit per resource unit, not number or resource units.
Why have spot H100 prices been going down then? It was roughly $3/hr a year ago and now it is closer to $2.2/hr.
Reminds me of the fiber boom
"The later versions of M365 copilot (or whatever it is called today) are wildly better than the original ones."
I find AI agents work very poorly within the Microsoft ecosystem. They can generate great HTML documents (because it's an open source format maybe?) but for word documents, the formatting is so poor I'd had to turn it off and just do things manually.
Opposing anecdote: I got consistent performance out of Grok and Qwen (17 providers on Openrouter) throughout the day but Gemini gets slow and dumb at times.
BlackRock just bought a data center for 40bn last week. BlackRock. The animals that bought all the housing once. They must be stupid I guess.
BlackRock did not buy “a” data center, it bought a data center company with 78 data centers. I have no comment on whether or not it was a good deal, but your framing is silly.