Wouldn't improving LLM efficiency make them even more useful across the board, then they can enjoy the nice economies of scale?
The plan is to have LLM working completely autonomously, in that case, the more resources you have, the better.
Perhaps people will use local LLM to ask questions, or coders use them for their personal projects, but that's not where the real money is.
The problem is that if the AI companies pass through the actual costs they're incurring, then charges to those companies will >10x.
If the companies don't see that kind of value (so LLMs don't become dramatically better in some kind of quantum leap from where they are now), they won't want to pay those costs. Already, most AI projects in corporations tend to fail.
If the efficiency of LLMs gets 10x better, then either corporations will "private cloud" their own AI or start using competitors that aren't carrying those kinds of debt loads from the "gold rush" phase.
If it's efficient enough you just run it all locally & screw all the rent seekers who want to tell you how you can't use their model & who will sell and misuse all your data they capture.
Truly, people have been saying this since 2017 and the Apple Neural Engine has proven them wrong time immemorial.
Apple's own desktops, with the fastest Apple Silicon GPUs and TDPs 20x higher than an iPhone still can't compete for real-world datacenter use even with RDNA clustering. Apple's GPGPU architecture is behind AMD at this point, there's a reason why Apple Intelligence is critically reliant on Nvidia and Google to provide inference backends.
Given how heavily subsidized it is at the moment, the efficiency isn’t as important. Typically efficiency would give you more at lower cost, but with token prices so removed from actual cost that plays less of a role here.
If the inference gets an order of magnitude cheaper, labs can afford to subsidise an order of magnitude more usage for the same marketing cost. So that part of usage will, if not accelerate with efficiency, at least still grow linearly with it. And there is a substantial amount of usage at or above true costs - everyone using a 3P harness, everyone on enterprise contracts, and everyone self-hosting an open weights model in a 3P cloud.
I assume the reference was to the Jevons paradox, as described in Jevons' 1865 book, "The Coal Question". Watt's steam engine massively increased the efficiency of coal in steam engines, which increased the use of coal fired steam engines, which increased coal consumption.
The (any!) comparrison to photovoltaics is
not acurate.Photovoltaics (PV) are primary energy producing infrastructure that produces its own fuel and is now verticly integrated into it's own supply chain, nothing other than life itself posseses this atribute.
AI, is exceptionaly likely to work in exactly the opposite fashion and take its host out as it goes down.
PV had to prove that it was truely indispensable and also prove to have realistic prospects for improvement and volume production before investments were made, and then prices came down.
AI has proven that it burns more money faster than anything else, ever.
I will admit that I am an early adopter of solar, but an AI refusenic, but still there is no reasonable comparison of AI and
anything outside of religion.
We haven't even started with a lot of things were we need a lot more compute:
Your real personal agent which knows you and helps you like "good morning elmer2, your calendar invite for dinner is today, you will need to leave at 18:18 if you want to use your normal public transport route per train. I put an alarm in your phone for you"
Agents to agents
Agentic teams.
Finetuned models for everything like Java/spanish coding model.
Very long term research like multiply hours or days or weeks and plenty of these in parallel.
I disagree in a way, part of the reason they can't really succeed at the moment is because it's way too expensive to really deploy at scale for most companies, but even for those AI companies themselves. If they can make business access subsidized/cheap the same way pro/plus/max/whatever plan are for regular users while still being profitable, this can work out. The other solution is if they do reach that "it's so super smart it's reinventing the world every day", but that one is much more of a maybe possibly one day.
What they can't do is the rug pull of pricing like Fable did, hoping for profitability while playing the "it's so super smart" card. It's very profitable, but customer will be very happy to leave for cheaper pasture and that's why the recent news about this or that cheaper chinese models make headlines.
Essentially, the rush now is "if I make it a boring profitable company I'm not worth a trillion AND i'm overshadowed that plays the singularity card even if they're bullshitting"
You do realize "subsidizing" means charging less for something than it costs to provide, right? So they'll lose a dollar on every sale, but they'll make up for it in volume? E2E is usually where the profit comes from. If they're subsidizing getting regular users on board (pro/plus/max), and they're subsidizing to get businesses on board (massive deploys), where can the profit possibly come from without a pricing rug pull?
I do, my point was answering to the "if it comes so cheap that" they would stop losing money of that, they would still need to subsidize for acquisition or some big clients or for rush times. It's the all-you-can-eat-buffet strategy.
I'm not saying I see them going that way or that I would, but at least THAT would possibly work.
Thank you for the clarification. I figured I was missing something in your meaning. Although "if it comes so cheap that..." means it will probably come cheap for other providers, and the margins wouldn't be there in the end because a price increase to take advantage of those big clients / rush times might lose the clients. I think we agree that their chances of becoming profitable don't look great.
How about we regulate private corporations so they can't take a "gamble" that's equivalent to a private company giving everyone ferraris on the idea that they will eventually all become formula 1 drivers and give back 10x the ferrari's cost?
Even better, that "gamble" will have to be rescued by taxpayer money.
Jevon's Paradox ("As efficiency of resource use increases, usage of the resource increases") says otherwise. One things become more efficient, we can use them in lots of ways that would not have been viable before, driving up usage.
Wouldn't improving LLM efficiency make them even more useful across the board, then they can enjoy the nice economies of scale?
The plan is to have LLM working completely autonomously, in that case, the more resources you have, the better. Perhaps people will use local LLM to ask questions, or coders use them for their personal projects, but that's not where the real money is.
The problem is that if the AI companies pass through the actual costs they're incurring, then charges to those companies will >10x.
If the companies don't see that kind of value (so LLMs don't become dramatically better in some kind of quantum leap from where they are now), they won't want to pay those costs. Already, most AI projects in corporations tend to fail.
If the efficiency of LLMs gets 10x better, then either corporations will "private cloud" their own AI or start using competitors that aren't carrying those kinds of debt loads from the "gold rush" phase.
If it's efficient enough you just run it all locally & screw all the rent seekers who want to tell you how you can't use their model & who will sell and misuse all your data they capture.
What happens when the AI god doesn't appear, and these models plateau in regimes supportable with high-end laptops?
If apple puts an inference SOC in their phone, the datacenters are all dead.
Truly, people have been saying this since 2017 and the Apple Neural Engine has proven them wrong time immemorial.
Apple's own desktops, with the fastest Apple Silicon GPUs and TDPs 20x higher than an iPhone still can't compete for real-world datacenter use even with RDNA clustering. Apple's GPGPU architecture is behind AMD at this point, there's a reason why Apple Intelligence is critically reliant on Nvidia and Google to provide inference backends.
Or: increasing resource efficiency may encourage even more usage, as happened with coal, oil and photovoltaics.
https://en.wikipedia.org/wiki/Jevons_paradox
Given how heavily subsidized it is at the moment, the efficiency isn’t as important. Typically efficiency would give you more at lower cost, but with token prices so removed from actual cost that plays less of a role here.
If the inference gets an order of magnitude cheaper, labs can afford to subsidise an order of magnitude more usage for the same marketing cost. So that part of usage will, if not accelerate with efficiency, at least still grow linearly with it. And there is a substantial amount of usage at or above true costs - everyone using a 3P harness, everyone on enterprise contracts, and everyone self-hosting an open weights model in a 3P cloud.
They improved the efficiency of coal?
Absolutely. Early engines were so inefficient that they could basically only be used in coal mines.
Massively. Extracting more useful energy from coal has always been a goal.
Very much unlike with software. Where the goal for long while is to burn as many resources as possible on end user devices.
I assume the reference was to the Jevons paradox, as described in Jevons' 1865 book, "The Coal Question". Watt's steam engine massively increased the efficiency of coal in steam engines, which increased the use of coal fired steam engines, which increased coal consumption.
Of coal use, yes. That's what inspired this: https://en.wikipedia.org/wiki/Jevons_paradox
The (any!) comparrison to photovoltaics is not acurate.Photovoltaics (PV) are primary energy producing infrastructure that produces its own fuel and is now verticly integrated into it's own supply chain, nothing other than life itself posseses this atribute. AI, is exceptionaly likely to work in exactly the opposite fashion and take its host out as it goes down.
When PVs got cheaper, more of them were sold.
PV had to prove that it was truely indispensable and also prove to have realistic prospects for improvement and volume production before investments were made, and then prices came down. AI has proven that it burns more money faster than anything else, ever. I will admit that I am an early adopter of solar, but an AI refusenic, but still there is no reasonable comparison of AI and anything outside of religion.
[flagged]
We haven't even started with a lot of things were we need a lot more compute:
Your real personal agent which knows you and helps you like "good morning elmer2, your calendar invite for dinner is today, you will need to leave at 18:18 if you want to use your normal public transport route per train. I put an alarm in your phone for you"
Agents to agents
Agentic teams.
Finetuned models for everything like Java/spanish coding model.
Very long term research like multiply hours or days or weeks and plenty of these in parallel.
This is going to sound a bit facetious, but I'm almost certain the Gmail / maps integration on my Samsung galaxy in 2014 did that.
I disagree in a way, part of the reason they can't really succeed at the moment is because it's way too expensive to really deploy at scale for most companies, but even for those AI companies themselves. If they can make business access subsidized/cheap the same way pro/plus/max/whatever plan are for regular users while still being profitable, this can work out. The other solution is if they do reach that "it's so super smart it's reinventing the world every day", but that one is much more of a maybe possibly one day.
What they can't do is the rug pull of pricing like Fable did, hoping for profitability while playing the "it's so super smart" card. It's very profitable, but customer will be very happy to leave for cheaper pasture and that's why the recent news about this or that cheaper chinese models make headlines.
Essentially, the rush now is "if I make it a boring profitable company I'm not worth a trillion AND i'm overshadowed that plays the singularity card even if they're bullshitting"
You do realize "subsidizing" means charging less for something than it costs to provide, right? So they'll lose a dollar on every sale, but they'll make up for it in volume? E2E is usually where the profit comes from. If they're subsidizing getting regular users on board (pro/plus/max), and they're subsidizing to get businesses on board (massive deploys), where can the profit possibly come from without a pricing rug pull?
I do, my point was answering to the "if it comes so cheap that" they would stop losing money of that, they would still need to subsidize for acquisition or some big clients or for rush times. It's the all-you-can-eat-buffet strategy.
I'm not saying I see them going that way or that I would, but at least THAT would possibly work.
Thank you for the clarification. I figured I was missing something in your meaning. Although "if it comes so cheap that..." means it will probably come cheap for other providers, and the margins wouldn't be there in the end because a price increase to take advantage of those big clients / rush times might lose the clients. I think we agree that their chances of becoming profitable don't look great.
How about we regulate private corporations so they can't take a "gamble" that's equivalent to a private company giving everyone ferraris on the idea that they will eventually all become formula 1 drivers and give back 10x the ferrari's cost?
Even better, that "gamble" will have to be rescued by taxpayer money.
no - see Jevon's Paradox
Jevon's Paradox ("As efficiency of resource use increases, usage of the resource increases") says otherwise. One things become more efficient, we can use them in lots of ways that would not have been viable before, driving up usage.
Don’t worry, they’ll get bailed out
[dead]