I don’t know why everyone’s confident that the investments won’t pay off. Every such post in HN is such a way. 800 million weekly active users from ChatGPT implies that people actually like LLM’s and likely the growth will still keep increasing. Every signal points to this - so investment in data centers make sense?
History doesn't repeat itself, but it rhymes. Callbacks to the infrastructure laid out during dotcom bubble, i.e Cisco, and all of the networking infrastructure. Likewise, internal memos at Oracle indicate they're losing money on their hardware. Anecdotal and hand wavy; but there are plenty of signals out there that it won't pay off. I'm not arguing one way or the other, but there are plenty of arguments for it to not succeed in a way that's required to justify the mind boggling growth we've been experiencing.
The mind boggling growth you should also focus on is consumer demand of LLMs. Can you account for it and then re-analyse?
Surely, Oracle could make up for its unit losses with volume, just like SNL's "Change Bank"
Lol CityWide Change Bank. I was nervous because I was running out of change, then I saw a CityWide Change Bank up ahead. She went to CityWide Change Bank to get exact change for the toll, "to save time".
why do you assume unit losses? they can just price it a bit higher
The technology obviously has merrits. When the bubble pops LLMs won't go away, just like the dotcom crash didn't spell the end of the internet.
The issue isn't that there aren't good business models or value creation, it's that anything related to AI currently has valuations that are unsustainably high given the current limits of the technology. That leads to economic activity that just couldn't exist without those valuations. And once the hype cools down the valuations will go closer to reality, leaving a lot of companies unviable, and many more will have to severely cut back spending to remain viable.
Or maybe the entire AI market pulls a Tesla and just stays at valuations that aren't justified by normal market fundamentals. Or maybe the technology adapts fast enough to keep up with the hype and can actually deliver on everything that's promised. This doesn't have to come down, it's just very likely that it will.
> internal memos at Oracle indicate they're losing money on their hardware
That doesn't mean much does it? Oracle is a huge company. Not just a cloud either. Companies often offer discounts or promotions; so? There could be plenty of managed services, managed databases, CRM and many more that make up for it.
Whilst I'm not sure if Oracle's stock price is right - the memo was more like a way to pressure the stock down for whatever reason.
History doesn't necessarily repeat nor rhyme. For 15 years now there have been people wrongly calling a bubble. The justifications change but looking back we can say they were wrong - or at least, a "bubble" that never pops and people get bored of talking about might have just been actual, real economic growth.
September 2020: 2020 Tech Stock Bubble (Sunpointe Investments, tech in general)
https://sunpointeinvestments.com/2020-tech-stock-bubble/
August 2017: When Will The Tech Bubble Burst?" (NY Times)
https://www.nytimes.com/2017/08/05/opinion/sunday/when-will-...
March 2015: Why This Tech Bubble is Worse Than the Tech Bubble of 2000 (Mark Cuban, bubble is social media)
https://blogmaverick.com/2015/03/04/why-this-tech-bubble-is-...
May 2011: The New Tech Bubble (Economist, bubble is "web companies")
https://memex.naughtons.org/where-angels-dare-to-tread-the-n...
And of course I haven't even bothered listing all the people who said cryptocurrency is a bubble. That's 15+ years of continuous bubble-calling.
At some point you have to say that if the thing supposedly inflating the tech bubble changes four or five times over a period that lasts a big chunk of a century, then maybe it's not a bubble but simply that economic growth comes from only two sources: a bigger population and technological progress. If technological progress becomes concentrated in a "tech industry" then it's inevitable that people will start claiming there is a "tech bubble" even if that doesn't make much sense as a concept. It's sort like claiming there's a "progress bubble". I mean, sure, there can and will be bankruptcies and retrenchments, as there always are at the frontier of progress. But that doesn't mean there's going to be a mega-collapse.
Similarly, economists have predicted 9 of the past 7 recessions.
Building the datacentres may be massively "productive" in GDP terms (servers! GPU! Power plants! Electrical inspection!), but when they're completed, that activity will cease. Datacentres don't employ many people directly and mostly consume only electricity.
So if the action of datacentre building shows up as essentially the only GDP growth, but what later happens in the datacentres fails to take its place or exceed it, there will be a dip.
Whether LLMs grinding away can prop up all GDP growth from now on remains to be seen. People use them when they're free, but people also collected AOL discs for tree decorations because they were free.
You don’t seem to have got my point. You say the investment won’t pay off but why not? This is quite a big assumption to make considering there’s huge evidence that people like LLMs.
I'm not saying they won't (the "if" is an important word in the sentence), it's just that datacentre growth is not the same as AI growth. It's related, but it is, by definition, temporary as datacentres are soon completed.
There's obviously evidence people use LLMs. That's not necessarily the same as people paying a noticeable fraction of all their money to use them in perpetuity. And even if "normal" people do start taking out $50 subscriptions as a matter of course, commoditisation could push that price down as could "dumping" of cheap models from overseas. A breakthrough in being able to run "good enough" models very cheaply, or even locally, would also make expensive cloud AI subscriptions a hard sell. And expensive subscriptions are the only way this pans out.
It hasn't yet been shown that AI is a gas that will fill all the available capacity, and keep it filled. If bread were 10 times cheaper, would you buy 10x as much? That has more or less happened to food availability in the West over the last 200 years and OpenBread and BunVidia don't dominate the economy.
None of that is sure to happen, and maybe the AI hype train is right and huge expensive LLMs specifically drive a gigantic productivity boom¹ and are worth, say, 0.2*GDP forever. But if it isn't, and it turns out $5 a month gets you all people actually need, it's going to be untidy.
¹: in which case, why is GDP not growing from the AI we already have?
It is very interesting to see two completely different positions on the economics AI but come to the same conclusion that AI is a bubble
1. LLM's are so economically unfeasible that companies won't be able to make a profit and investing in datacenters will turn out to be a bad bet because AI companies themselves are a bubble
2. LLM's will become so cheap that datacenters will be useless and people will just use local models so investing in datacenters is a bad bet
I see both positions in this thread so which one is true?
It's a spectrum of possibilities upon which, as yet, no one is sure where it will land. Invested proponents say it will eat the world. Sceptics say it's all a crock of shit. Anyone who says they know the answer, doesn't.
The two positions there aren't really different, they're mostly that the profitability of AI can be eroded from several sides. One: the cost to run (power and hardware) being high and bring unable to recover it from revenue. Or two, commoditisation and efficiencies (which can also be operational convenience rather than only about power) driving down costs and therefore also revenue, and being unable to compensate by selling more AI more cheaply. Three: AI didn't actually help as many people make money as hoped and thus they don't want to pay, also depressing revenue.
In the middle is the three-axis happy AI place where costs are not too high, but also AI is too hard to have someone else do it cheaper, and it's useful enough to be paid for.
My guess is AI ending up roughly as impactful overall as cloud computing. A big industry, makes a lot of money, touches and enables very many businesses but hasn't replaced the entire economy, profitable especially if you can stake out a moat, with low-margin battlegrounds for the price-sensitive.
I agree with all your points. But I still do feel it is interesting that the vibe is on both ends of the opposite spectrum - both highly efficient and highly expensive.
Maybe it just works out like CPUs or as you said cloud computing? CPU's got cheaper and demand increased and more people use them but everyone still made a profit.
It just goes to show how little information there really is about what will happen here. Literally no one knows. It's uncharted territory, but it's filled with map-sellers.
Another good example is railways in the US. That was an huge, huge boom, around a fifth of GDP. No one knew what the railway-based economy of the 1900s would look like but it was surely going to be spectacular. All that money! The speed! The cargo! All those people! Railways absolutely were a commerce multiplier, and made stacks of revenue very quickly and got investment from around the world to build build build. But, eventually, the (over)building was done, there were bankruptcies and consolidations and it ultimately did not become the dominant industry. And yet, it's still a big industry that enables a lot of other economic activity. Trains are still expensive to operate, but moving goods is pretty cheap. Obviously there's a natural physical monopoly at play there that AI doesn't have so again, who knows.
Which leads to another thought the AI investors, both commercial and national, maybe should eventually have: is there an automobile or airliner to their railway?
People like LLMs, will they pay for LLMs to offset the investments? Will they pay enough so these companies can stay afloat when the slush funds inevitably run dry?
Will LLMs create a significant shift in productivity where its usage will create enough overall value to the economy to justify the hoovering of capital from other industries?
Those are the unknowns, I don't think many people are saying that LLMs have no value like NFTs, it's that the money being pushed onto this novelty is such an absurd amount that it might pull down everything else if/when it's discovered that there won't be enough value generated to compensate for trillions of USD in investments. Hence the comparison to the dotcom bubble, we came out of that with the infrastructure for the current internet even though it was painful for a lot of people when it crashed, will we have a 2nd internet-esque revolution after this whole thing crashes?
The technology is definitely valuable, and quite fantastic from a technical standpoint, is it going to lift all the other boats in the rest of the economy like the internet did though? No one can tell that yet.
You don't think it's a bigger assumption that plowing hundreds of billions of dollars into a novel and often misunderstood technology is going to net out positive? We're not talking "people like feeds of cat photos" money and risk here, we're talking about a bubble with the potential to tank the entire economy. On top of that, it's also a higher disruptive technology that should it ramp as quickly as it might will lead to massive amounts of societal strife at a time where we're already stretched a little thin to say the least. "What do you mean Credit Default Swaps don't work? They've been functioning perfectly well these past few years and so far the impacts have only been positive. Tons of people are getting mortgages on houses they weren't in the past, it's the American dream!"
The question is if it is economically feasible. People can enjoy generating funny AI images to share with their friends, but it might not be economically feasible to invest $100B to give them this toy. There is a question of how much value using GenAI generates for the economy.
The cost of LLMs have gone down over 30 times in the last 1-2 years. At what point would you think it is economically feasible? I think this a question to ask so that we can tackle the fundamental economics of LLMs.
It becomes feasible when people are willing to pay more than it costs to run it. But I think this will be a pretty uphill battle, as many use cases are hard to monetize (eg proofreading) and for many use cases you will feel pressure from smaller models (you don't need the most expensive SOTA model to generate an email). There is probably just a very limited amount of use cases that are in the goldilocks zone with their difficulty so that people will be willing to pay for them, AND AI is able to solve them. I think programming might be one of them.
> It becomes feasible when people are willing to pay more than it costs to run it.
This is what I want to challenge. At what point do you think people will pay more than it costs? Lets try to come up with a number because the price of LLMs have dropped more than 30 times in the last 2 years.
It may continue to drop and AI companies will continue to be in loss because the new things will be unlocked due to new efficiences and the same debate over LLM economics will continue.
I think it is already profitable and people are more than willing to pay for the actual costs.
> I think it is already profitable and people are more than willing to pay for the actual costs.
If people are willing to pay for the costs, where are the profitable AI companies?
They don’t make profit because they invest in research and development.
We would also need to know how many people are willing to pay for it, on a consumer level. Additionally business will need to see real ROI for using it.
At the moment everyone is trying their best to implement it, but it remains to be clear if it actually increases a company's profitability. Time will tell, and I think there are a lot of things obfuscating the reality right now that the market will eventually make clear.
Additionally the economics of training new models may not work out, another detail that's currently obfuscated by venture capital footing the bill.
And yet 90% of AI companies do not break a profit. If it was economically feasible, big corps would start cashing in immediately and halt development. They have gone deep with AI so now must go deeper to get to that cashing-in point.
I don't see why they would want to halt development. The industry isn't at a stagnant stage when PE comes in and sucks all the joy out of things.
"People actually like LLM's"
People also like pizza. How many million weekly active consumers of pizza? how about rice?
Really what people like here is cheap stuff and having a job that pays money to buy it. chatgpt so far loses boatloads of money. Soon they jack up prices, add adds, and people realize that it was all trained on them & threatens their job. So really right now chatgpt is sweating hard to make itself too big to fail.
https://news.ycombinator.com/item?id=45511368
> 800 million weekly active users from ChatGPT implies that people actually like LLM’s
800m total users, 25m paying customers... Most people use free accounts and would likely never pay any substantial amount of money for them
https://www.theverge.com/openai/640894/chatgpt-has-hit-20-mi...
Forget profitability, the major Western LLM providers still have profoundly negative gross margins
I think it will only be economically sound as a business if you're Google and can start serving ads OR when we switch over from GPUs to wildly more efficient TPUs/ASICs
All the data center CapEx is going into compute that will be obsolete once that happens
We have hit a plateau for many months for the performance of LLMs. Anthropic recently released 4.5, and while it improved on some contexts, it failed to make a commit message for me a few times on a workflow I had. 3.5 to 4 had close to zero failure rates on this workflow and it was surprising to see 4.5 fail. It seems that gains in certain benchmarks will affect quality elsewhere.
LLMs are very useful, I can’t see myself walking back to the old way of doing things. But the amounts invested expect major breakthrough that we are not anywhere near. It’s a gamble and that’s what innovation is; but you gamble on a small portion of your wealth. Not your house and certainly you do not gamble a huge country like the US on a single thing.
They like them when they’re free and/or cheap. But will that be sustainable? The answer to that question is far less certain. Maybe Ads will save them, maybe royalties, maybe price hikes. But it’s far from certain at least.
I really don’t get this. People actually pay and use and enjoy llms. My company has paid for Gemini and ChatGPT subscriptions and people actually use them.
Why do you automatically assume that people won’t pay for it?
Because they probably would have to pay a lot more to make this profitable at the levels current valuations and investments indicate. And the barrier for private persons to pay is much larger than for companies. I don't think anyone has a really solid handle on the economics here yet, as the field is changing very quickly.
But there is a big difference here compared to most software companies. The product does cost significant money per additional customer and usage.
There is a real product here. And you can likely earn money with it. But the question is "how much money?", and whether these huge data center investments will actually pay off.
> Because they probably would have to pay a lot more to make this profitable at the levels current valuations and investments indicate.
I keep hearing this but this is very unlikely to be true. The cost of LLMs have gone down by more than 30 times in the past 1 year. How much more should it go down until you consider it economically feasible?
Why are they building so many data centers then? That is all cost that has to be earned back. And using them as agents creates much higher costs per interactions than just chatting. We also don't know if the current prices are in any way economical, or how they are related to actual development and interference costs.
Do you think people should have not invested in data centers because of moores law that also applies for cpus? Same mechanics applies there - turns out that when things get efficient, demand increases and more possibilities are unlocked.
When Moore's Law was still effective, did you ask why people produced chips?
Because all numbers point towards it being incredibly far away from being profitable? We also pay for Google Workspace and at 10 euro a month we get Gemini Pro. So while we might pay for it, it’s more of a free addon, we would’ve paid 10 without it too.
You can also do a simple analysis on the Anthropic Max plan and how it successively gets more and more limited, they don’t have the OpenAI VC flow to burn so I believe it’s a indicator of what’s to come, and I could of course be wrong.
It’s not profitable because of massive reinvestments to r and d.
If you want to question to on the fundamental economics of LLm themselves then how efficient should LLMs get till you decide that it’s cheap enough to be economically viable? 2 times more efficient? 10 times? It has already gotten more than 30 times over last 2 years.
And Claude is more expensive than ever, efficiency gains and all. Those investments doesn’t necessarily pay off, and historical performance is not indicative of future ditto.
I don’t think it’s a matter of efficiency at current pricing but increased pricing. It would be a lot more sane if the use cases became more advanced and less people used them, because building enormous data centers to house NVIDIA hardware so that people can chat their way to a recipe for chocolate cake is societal insanity.
> And Claude is more expensive than ever, efficiency gains and all
This is not true for any LLM and not just Claude.
> I don’t think it’s a matter of efficiency at current pricing but increased pricing.
I don't know what this means - efficiency determines price.
> It would be a lot more sane if the use cases became more advanced and less people used them, because building enormous data centers to house NVIDIA hardware so that people can chat their way to a recipe for chocolate cake is societal insanity.
Do you think same thing could have been said during the internet boom? "It would be more sane if the use cases become more advanced and less people used them, because building enormous data centers to house INTEL hardware so that people can use AOL is societal insanity".
Weird how Sonnet 3.7 cost the same (when released) as Sonnet 4.5. That is with all those efficiency gains you speak about. 4.5 is even more expensive on bigger prompts.
Efficiency doesn’t determine price, companies does. Efficiencies tend to give more returns, not lower prices.
Internet scaled very well, AI hasn’t so far. You can have millions of users on a single machine doing their business, you need a lot of square footage for millions of users working with LLM’s. It’s not even in the same ballpark.
Did we build many single company data centers the scale of manhattan before AI?
> Weird how Sonnet 3.7 cost the same (when released) as Sonnet 4.5
Then I think we agree that while the cost remained the same, the performance dramatically increased.
FWIW Sonnet 3.7 costs 2.5x as much as GPT-5 while also being slightly worse.
Well with a 30x increase in efficiency and far from 30x more performance that would be a price increase in this context, the efficiencies clearly doesn’t trickle down to customers.
As for OpenAI I don’t think anyone is working on the API side of things since GPT-5 has had months of extreme latency issues.
Forget profitability, the major Western LLM providers still have profoundly negative gross margins. What would that 800M userbase look like if free tier users had to pay the bill for inference and training costs?
I think it will only be economically sound as a business if you're Google and can start serving ads OR when we switch over from GPUs to wildly more efficient TPUs/ASICs
All the data center CapEx is going into compute that will be obsolete once that happens
Yeah, I don't think ChatGPT is going to take all our jobs, but it might replace Google for a lot of searches.
That alone will be a monumental shakeup for the industry.
Imagine how many millions would drive a Ferrari if they gave them away for free
> Imagine how many millions would drive a Ferrari if they gave them away for free
0.
Ferrari is a luxury sports brand. What's the point of it if it flooded the streets?
Good looking, powerful, reliable car?
> Good looking, powerful, reliable car?
How to say you don't own a Ferrari without saying you don't own a Ferrari.
It’s not true that people are only using it because it’s free.
It’s actually quite interesting to see these contradictory positions play out:
1. LLMs are useless and everyone is making a stupid bet on it. The users of llms are fooled into using it and the companies are fooled into betting on it
2. Llms are getting so cheap that the investments into data centers won’t pay off because apparently they will get good enough to run on your phone
3. Llms are bad and they are bad for environment, bad for the brain, bad because they displace workers and bad because they make rich people richer
4. AI is only kept up because there’s a conspiracy to keep it propped up by Nvidia, oracle, OpenAI (something something circular economy)
5. AI is so powerful that it should not be built or humanity would go extinct
It is true that none of the LLM providers are profitable though, so there is some number above free that they need to charge and I am not convinced that number is compelling
None of LLM providers being profitable is exactly the situation I would expect. Them being profitable is so absurd on the contrary! Why wouldn't they put the money back into R&D and marketing?
I'm not well versed with the accountant terminology, whatever the word is to describe the operating cost, I am not convinced consumers will ever pay enough to cover those costs
Do you think if LLM's become 10 times more efficient it might covert he costs? What efficiency increase would you think is enough?
It's a competitive environment, no way the data centers manage to capture that 10x efficiency improvement. There would be an expectation of 10x reduced prices, because someone else is offering that.
The problem I see as someone who has implemented a bunch of AI solutions in a range of markets, the quality isn't good enough yet to even think about efficiency - even if the current AI is 100x more efficient it still wouldn't be worth paying for because it doesn't deliver reliable and trustable results...
A) Huge straw man, since it isn't the same people making those points. None of those need the other to be true to cause issues, they are independent concerns.
B) You're missing a few things like:
1. The hardware overhang of edge compute (especially phones) may make the centralized compute investments irrelevant as more efficient LLMs (or whatever replaces them) are released.
2. Hardware depreciates quickly. Are these massive data centers really going to earn their money back before a more efficient architecture makes them obsolete? Look at all the NPUs on phones which are useless with most current LLMs due to insufficient RAM. Maybe analogue compute takes off, or giant FPGAs, which can do on a single board what is done with a rack at the moment. We are nowhere near a stable model architecture, or stable optimal compute architecture. Follow the trajectory of bitcoin and etherium mining here to see what we can expect.
3. How does one company earn back their R&D when the moment it is released, competition puts out comparable models within 6 months, possibly by using the very service that was provided to generate training data.