> hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments.
I think everyone has known that inference is profitable overall for the major providers for a while (I'm sure not all user accounts, but in aggregate).
The major providers are unprofitable after you add in the training and research costs and other overheads.
This is why the open source models are such a threat if they become good enough - it will destroy the business model of just providing raw chat/intelligence.
Arguably that's also why a few of these providers are trying to diversify into whole-platform approaches rather than just relying on having their whole thing be a great model.
There has always been a lot of evidence that prices are only going down. The fact that you could already get DeepSeek or GLM inference from 20+ providers on openrouter.ai was already sufficient evidence that inference was profitable and being commoditized.
People that want to believe something different as a coping mechanism will not be swayed by this evidence either.
AI will end up being a commodity. That's also why you already see the first rounds of protectionism under the guise of national security. Obviously they do have a point but that point works against them just as easily. Anybody that integrates unreviewed model output into their workflow is asking for trouble.
non frontier intelligence will be a commodity. Frontier intelligence will be a high margin luxury good. this is how most markets work
Yesterday’s frontier is today’s commodity, so I don't think frontier AI is out of reach. With computing and solar energy costs continuing to drop, the current AI supply shortages are just a temporary blip. We’ll probably land right back in standard boom-and-bust cycles.
Either frontier AI will continue to get better steadily, until it's better than people at basically everything, and very weird things start to happen, or, it plateaus and the open weights models will eventually catch up.
There's a third possibility, which to me is also the most plausible. Models will keep getting better and better within certain domains, while never catching up with humans when it comes to other type of tasks. Like that elusive quality-without-a-name some have called "taste". Or knowing what UX is most intuitive to humans. Really, anything where we have to tap into our common humanity and our human experience in order to be of help to other humans and generally to life on earth. Which is an all-important skill, since the flowering of life on the earth is (or should be) the whole point of anything we do, after all.
We've spent a couple of millennia documenting that elusive thing called 'taste' in word, sound, image, lately in moving images. I'm pretty sure that even if you didn't have taste you could make a good simulation based on all that input.
I don't know, that sounds like a question of taste. :)
It can remix things people have already documented but can it synthesize an entirely new thing?
to synthesize an entirely new thing you have to start form the big bang. Everything else is derivative.
So what you're saying is that since there are so many examples of what humans have preferred in the past for these models to be trained on, surely there will come a time when they will know even better than us what we want and what we like.
... as long as they still care about what we want and what we like.
Everyone knows the path followed by every single technology mankind has ever developed - including the incredible run that was Moore's law. Sure, some will think "This time is different" - but is it?
It doesn't need to run indefinitely. I don't think anyone sane thinks it's going to run indefinitely. It just needs to run long enough. Moore's law was what, 50 years? If AI keeps improving at the rate it currently is for 20 years, the world will be very different when that curve flattens out, in a way that makes most of our current concerns are entirely irrelevant.
Those cycles have shortened considerably. Just look at the lifespan of media, integrated circuits, software, products. There have been so many big model releases just in the last three months I've lost count. We're now closer to weeks between steps.
Maybe if China stops releasing capable models the pace will slow down a little, but they have lots of incentives not to.
It is, because just like you mentioned, there's no exponential physics to back up growth. LLMs have to stop scaling up compute, because there's only so much resources.
But even if we think back to the previous decade or two (sans AI), most of tech has been a a red queen's race - developing and improving stuff, without any real substantial results.
Microservices and orchestration are not better than old monoliths. Electron apps are not better than native. Most programs people actually use have decades-old codebases.
Video games of today don't look substantially better or play different than ones from 15 years ago.
But it is not a constant point of "this % of people needs the frontier intelligence".
As thje models get better, the % of the pie gets smaller and smaller. So higher and higher cost of getting frontier model out gets divided onto smaller and smaller customer base as other models catch up
Yes, this is the crunchy bit and we're already well underway on that curve. I haven't touched a frontier model in months because I can do with smaller ones without wondering/worrying when I'll get banned next because what I'm doing is dual use. Qwen was the the first time I thought oh, ok, so this is 90% of what I need. Then DS4 made that 95% and with GLM 5.3 it's somewhere between 99% and 100%. Now, I'm not trying to solve any Nobel prize winning stuff here but if that's the kind of customer that will remain I don't think the frontier model providers are going to like the next six months or so.
That's the crux of the issue. You think that GLM 5.3 is all you need.
However, the expectations for AI are not to make your software engineering work 100% faster.
It's, for example, to build software more or less end-to-end with minimal input and feedback from you.
Then next, robotics and boosting manufacturing. Not by 10% but let's say 200%.
Such are the ambitions. Can GLM 5.3 do it?
The crucial question is whether eg. Fable 6 can, and with what lead over the competition.
Open-weight models still have not beat February's Mythos.
I am actually doing robotics stuff and yes, GLM 5.3 can do it.
As for the manual feedback: I want that manual feedback, but if I let it rip it will do just fine all by itself, but then I have no clue what is going on under the hood.
At some point you have to let go of trying for full understanding and control, delegate to the AI, and unlock the real productivity boost.
I have no clue about the state of robotics.
For software, I know that Opus 5.5 can handle more than GPT 5.6 Sol. I am willing to pay for the difference rather than use the cheaper DeepSeek Flash.
> At some point you have to let go of trying for full understanding and control, delegate to the AI, and unlock the real productivity boost.
The prerequisite here is that you can trust it not to do something that gives criminals access to your bank account or allows users to consume an unlimited amount of services it costs you money to provide for free or bankrupts you through legal liability etc.
It's basically this:
"A computer can never be held accountable, therefore a computer must never make a management decision."
Notice what this is actually saying. It's not saying that the computer can't generate a given output. It's saying that if the computer makes a mistake and you don't bother to check, you're the one who has to eat it.
> "A computer can never be held accountable, therefore a computer must never make a management decision."
That's a great way of putting it. So far I always used 'you can outsource work but you can't outsource responsibility'.
> "A computer can never be held accountable, therefore a computer must never make a management decision."
Legendary quote from an internal 1979 IBM training document [1]
https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
I'm getting plenty of mileage without that and anything with a motor in it requires that you do your utmost best to ensure it works within the design envelope.
We aren't going to get useful humanoid robots while understanding the implementation fully.
If you can't leverage AI to build a safe and useful robot without fully understanding the implementation, the model is not good enough.
There are so many assumptions in these two lines that I don't know where to begin. I wish you best of luck.
The humanoid robot was an example.
My argument is that if safety and liability make the use of AI without fully understanding the implementation impossible, the model is arguably not good enough.
A long time ago we decided as a society that in order for an engineer to sign off on physical designs, they have to take legal liability for them.
No such process exists for a model to take a design seriously that can kill people.
For serious engineering work not only physical designs, software also.
When we manually tested nuclear power plant controlling software, each test case was printed out on paper and tested and then signed by the tester. A lot of paper was produced and archived for each software release. (Archived for as long as the software was running in any nuclear power plant).
Everyone understood, that in a case of significant accident at the nuclear power plant, accident analysis would be done and if the root cause was an software error, this could have possible legal consequences for the developers and testers. (Of course nuclear power plants have many independent and redundant hardware controlling mechanisms, but incorrectly working controlling software could damage the power plant in a very costly way).
With a larger project, I do not think there is a requirement that the engineer signing off understands every implementation detail.
I imagine that the requirement is that after testing, you are confident that the product is safe.
Respectfully: you are out of your depth. Maybe pick another subject?
It was you who shifted the conversation away from software to make an argument that "anything with a motor" requires full understanding of the software implementation, and that for this reason GLM 5.3 is good enough.
I think you're entirely wrong, and we can leave it at that, seeing that you're clearly unwilling to engage constructively.
The practice of deliberately engineering software without full understanding of the implementation, especially when it comes to robotics, should be illegal. Technology serves society alone and it should progress at a pace that is sustainable for adaption. Your excitement for a “singularity” does not outweigh someone else’s health and safety.
Nah man, no you don’t. There are very few absolutes in the world and that certainly isn’t one of them.
And over a increasingly small slice of time, as, due to the commodification of these things, the frontier businesses will see their customer base walk to the next one that does any better on their task.
This is probably ultimately unsustainable.
Every time they throw someone out with a message about how this request is definitely against their safety guidelines they are clinching the deal on someone running something local or switching to a provider that just lets them get on with their day. Their own fearmongering is going to be part of their undoing.
A high margin luxury good with huge r&d costs is a death spiral.
A high per-user r&d means high costs per user. So then users leave to save a buck, qnd the price goes up again for the remaining users.
Maybe it works out OK if you want to signal wealth - the high price is the selling point. But luxury pure utility is a hard sell to consumers, and for luxury B2B it's about high touch sales (e.g. Oracle).
Agreed, but I I wonder what fraction of tasks really require frontier performance, either in terms of sheer difficulty or where speed is a matter of necessity. For many jobs, it's increasingly a question of whether you'd rather have one filet mignon or 20 good cheeseburgers (all from the same cow).
I’m not sure if this counts as a nitpick, but the definition of frontier automatically includes the cheapest models
Who can and is willing pay top dollar for frontier human intelligence, and fully utilize it to actually make money?
The only answer I could come up with is hedge funds. Everyone else fails some part of my criteria.
The models will be a commodity, the resources to run the frontier will be the luxury good. This is already the case nearly.
The market for "frontier intelligence" will be too small to serve outside of certain government/military applications that exist in spite of market forces, in the same way that the market for small, portable nuclear reactors is exclusively limited to submarines and aircraft carriers. When I call a plumber, I don't need a frontier plumber, I just need someone who's baseline competent.
> market for small, portable nuclear reactors is exclusively limited to submarines and aircraft carriers
SMRs are being commercialized. Likewise nuclear cargo ships have been seriously studies. Nuclear reactor powered (not just RTG) satellites have been built.
I know this is beside your point, but I think if we figured out how to commercialize this, demand would come.
> I just need someone who's baseline competent
But why wouldn't you want (and pay) for the very best plumber? A frontier plumber will optimize the time their customers poop, pee, bathe, or do dishes 10x to 200x, and with their freed-up time,they'll outcompete those who don't pay for frontier plumbing!
You forgot the /s
Sir, I take the impending plumbing singularity very seriously, and wouldn't want anyone to be left behind. When plumbing hits recursive self-improvement, we may see toilets that shit themselves more efficiently than any man can, and finally attain ASI (Auto-Shitting Infrastructure).
I can see you do. I beseech you though to think about the consequences for other industries. For instance, the ratio of toilet paper:organic product will need to be maintained somehow and I suspect that if we follow your pathway to its logical conclusion that there will be a cellulose crisis unlike anything seen in the world since the USSR went bust.
The next move in my investment strategy is obvious though, invest to the hilt in PVC tubing and I thank you for this invaluable insight. Fortunately this forum has no visitors so we can rest secure in the fact that the pigs won't realize it is too late to avoid being slaughtered. Power to the bulls, after all it is their product more than anything that is responsible for the plumbing revolution.
there is definitely an asymptote
Non-frontier intelligence has become a commodity already.
And the success of things like ESP32 shows that good price + good enough is a powerful combination.
> AI will end up being a commodity
what does this even mean, the commoditization... like a thing you can buy from a gas station? AI commoditization is the rock bottoming of pricing.
Usually we end up with cartels in such cases, see OPEC, mafias etc. But we have China throwing sticks into the wheels of the US providers that are attempting to make the cartel and "pacing the frontier"
> Usually we end up with cartels in such cases, see OPEC, mafias etc.
Whether a cartel forms is independent of whether a product is a commodity. To the extent that commodity markets are more likely to form a cartel, it's only because it's harder for them to form a monopoly.
The general problem is that if your legal system allows competitors to buy each other then they have the incentive to do that instead of competing. This is why there are supposed to be antitrust laws, and why we get problems whenever they're not actually enforced. But in a typical commodity market, even buying the competition doesn't quite work because someone new can always enter the market. Congratulations, you've cornered the market on landscaping services... until some unemployed fellow who already owns a lawnmower and a truck starts advertising their services.
OPEC is kind of a special case because the product is entirely fungible but it's also finite and consumed by usage. Landscaping services aren't meaningfully finite, if demand increased then supply could match it under any plausible scenario. Copper is finite but it isn't destroyed by usage, so if someone had a lock on copper mining then people could still get it by recycling existing products.
Whereas oil is finite but then the end customer burns it. If Saudi Arabia cuts production, Germany can't make up for it by extracting more domestically because they haven't got enough in the ground, but they also can't capture the exhaust from their cars or power plants and turn it back into petroleum because the product is really energy and it takes more energy to do that than you get from burning it to begin with. So if the largest exporters get together, they control enough of the market to control prices and there is no real way for anyone else to enter the market with enough production capacity to change that.
Until you realize that the product is really energy and you can get out from under their thumb by switching to electric cars and generating electricity without burning fossil fuels.
A commodity is an undifferentiated good. I expect gasoline to be practically the same at each provider. AI will be similar.
The funny thing is the most lasting contribution OpenAI makes to this field could very well be the protocol for the endpoint.
Not too long in the future, I fully expect OpenAI to rescucitate the API copyright aspects of Google v Oracle to pad its revenue in OpenAI vs Everyone profitable
I wish them good luck. I don't doubt that they will try.
China is doing all the heavy lifting for the free market in AI right now. I did not have that in my 2020s playbook.
> what does this even mean, the commoditization...
Like a utility. Not 'ACME intelligence' but just 'Intelligence'. With multiple providers selling you intelligence that is 'good enough' by the token.
>Usually we end up with cartels in such cases, see OPEC, mafias etc.
No we don’t. That’s quite rare.
Look at the collapse of the solar market around 2010 as the price collapsed.
Look at the collapse of the steel industry in the US as outsourcing to Asia took over in the 70s-80s.
Look at the entire computing boom from 1990-2020. Memory and storage followed such a steady line down and to the right that people are struggling to operate compute businesses in a normal market.
E-bikes, scooters, electric cars, lithium ion batteries in general. Most food.
The list of these commoditized things that started out as areas of innovation and became just the marginal cost of production is enormous.
I’ve never heard anyone say raw inference was unprofitable. It wouldn’t even make sense, I can’t see a rug pull for renting out commodity GPUs.
People are saying _training + inference_ is unprofitable (I don’t have a dog in the race and don’t really care either way). That’s a lot of GPUs and salaries that aren’t generating revenue at any moment.
Personally, I would guess Anthropic and OAI are profitable. I just suspect that they’re a pretty normal level of profitable and not the absurd, near-alchemical “can turn dirt into gold” level of profitable their valuation seems to have priced in.
To add on, training + a highly paid American salary load + inference is unprofitable, if and especially if you have deepseek dropping weights and engines for anyone to run.
I don't see the average person being able to run DeepSeek's most competitive models though.
H100s are renting out at a few dollars per hour. An awful lot of people could run them if they wanted.
OpenRouter's DeepSeek provider is basically this. Instead of renting H100's for dollars per hour, you pay someone that arbitrages the $/hr to $/token conversion because you can't actually saturate an H100 for hours. It's still effectively the same as running it yourself because of the ability to change providers to someone else arbitraging the same thing, and that there's always the escape hatch of switching to actually renting the H100s yourself.
They don't have to, slightly above-average people can host in Asia for way cheaper than anthropic.
Why would it be cheaper? Electricity might be (slightly, possibly), but H100s aren't
It's a commodity, and research into how to make it good will be done in the open. What this means is that we should doubt the justifications offered by magnificent-N-whatever companies slurping up all investment dollars to take over the world by being the first to get "general" "super" intelligence. No, there won't be a runaway improvement. Sadly, AI is just a clone of our inner world. Getting interesting information into it is still going to be just as slow as it is in the regular world. You have to build, tinker, experiment, get data, get knowledge, etc. The only thing getting sped up is how much time needs to be allocated to thinking and translating.
there could still be a runaway improvement, but it also almost certainly goes with the decreasing prices
Isn't the modern economy already a runaway improvement? First writing and agriculture, the stone age became copper became bronze became iron became steel, and one Revolution after another - agricultural 1 and 2, industrial, information.
Looks like AI will be another revolution, and eventually, probably the largest.
There may be different thesis out there, but the fact of prices going down definitely doesn't help the case that the major AI labs are are overvalued and need to vastly increase their revenue to stay afloat amidst their contractual obligations.
I honestly don't think there are too many people arguing "AI is actually too expensive to take my job". "AI being too much of a commodity to make meaningful profits" could very well be the case though.
I have seen the former argued many times, though mostly not on HN. It is a cope meme I’ve seen mostly on Twitter and Bluesky, but some on HN and the Elixir forum as well.
They are literally hoping Anthropic & OpenAI go bankrupt (totally possible), and everyone goes back to using their meat brains for work (never happening).
In this context it just doesn’t matter if these companies are profitable after amortizing training cost, because deployment of open weight models that already exist will continue to disrupt economies for at least the next decade if all research ended tomorrow.
> The major providers are unprofitable after you add in the training and research costs and other overheads
The very short lives of all these models also factor in.
I think the funny thing is the more inference, the more tokens a specific version of a model spits out, actually does make it more profitable.
But somehow a lot of people truly think that each token generated increases the cost of OpenAI/Anthropic/... and shout bloody murder about "subsidies"
And I'm certain there are people out there that actually think the API prices of OpenAI and Anthropic somehow equal their electricity costs.
Nobody meets expectations of a 2 trillion dollar IPO selling a fungible commodity like tokens for old models.
And that's the rub - it doesn't matter if inference is profitable, the US labs have promised the whole kitchen sink, and taken serious investment on that pitch. They won't make it back in a cut-throat market for inference. They have to continue pitching the idea that only their models are worth using, for quality/safety/performance/etc reasons.
Which means continuing to develop and release new models. At huge expense.
Side note - while deepseek seems to be doing fine, I'll note that the emphasis on cost effectiveness is very much not a thing for the US labs right now... and it was for non-us labs. I'd be quite interested in knowing if anthropic is pulling in the same returns, or what the margin on their subscription plans actually is.
---
I'm already seeing this directly at work - lots of devs still happily use claude for development, but the products we're making aren't. They'll use kimi, glm, gpt-oss, etc... basically - if cloudflare is hosting it, we're exploring it (mainly for contractual/security reasons): https://developers.cloudflare.com/workers-ai/models/.
Because we are price sensitive, and we already have to do all the work of creating very robust evaluation frameworks for our products, so we might as well choose the cheapest model that meets our standards.
whiiiiich is why we're seeing things like Haiku and gpt-luna priced down near the open models at ~.10/mm in, ~.50/mm out. They've been forced to cut prices to keep commercial customers on their cheaper models instead of moving away.
This will continue. If inference is profitable and easy, prices will continue to erode. Go browse openrouter, plenty of providers happy to undercut the competition. Bigger shops with auditing requirements will avoid them, but smaller ones won't.
You know, it's not just cost-effectiveness, it's also iteration time and speed. DeepSeek V4.1 Flash is astoundingly fun to use because it's so quick. Tens of times faster than the standard Frontier models. With, in my experience, practically the same results. Because most of the trouble is not the intelligence of the model. It's the feedback from the world. The quicker you can get feedback from the world, the better your process is, irrespective of raw intelligence metrics of the AI you're using.
I 100% agree and am a big fan of DS V4.1 Flash in general for this reason, but I'll add my own caveat: Sol 6.1 isn't far behind on completion time for tasks because it's so damn efficient. I regularly get ~300 tokens/sec with DeepSeek Flash, but it spends the majority of that time thinking and executing slow commands one by one, whereas Sol thinks less and is much smarter about the commands it runs, resulting in far fewer turns to get the same thing done. For highly agentic tasks, DS might only be ~25-50% faster, not 500% as the TPS may suggest.
(But to be clear: 25-50% faster is still a big win in my book.)
+1; using Sol 6.1 in Ultrafast is noticeably faster than DS 4.1 Flash in real world tasks where marginal intelligence and token efficiency matters. Yes it costs more, but that’s a function of how valuable is the thing you are iterating on.
> Which means continuing to develop and release new models. At huge expense.
I guess my question is when do we say a model is "good enough"? For my use cases Opus 5.5 feels as if it is exactly at that point. And at that point of course one of the main factors I will evaluate against is price. So far I am happy with my (heavily subsidized) subscription, but if the time comes, I'll happily switch the inference API any of my agents are calling. So far I have no big lock in on any providers.
define "we".
Because again - I think lots of businesses are already taking a good look at models well off the "cutting edge" because they're good enough for what that business is trying to do. They're already there.
If you're doing categorization, tagging, form-filling, customer support routing, etc... you don't need anywhere close to cutting edge expensive models.
Hell - Jev being so popular is basically exactly that group going "what's the cheapest way we can make this decision".
Meanwhile, there are plenty of spaces where you still want real model improvement. But a lot of traditional business was basically process and paper pushing. You don't need a Phd level model for that, you need an intern level model.
---
But that's the trap for the large providers - intern level models are cheap and easily available with current open weights. It's going to be a very tight market, and margins will go down to essentially the floor (especially once we exit the current constrained-compute market).
Somewhat ironically - the technology itself makes switching costs basically zero. It's not just that there's no big lock in, AI is like an anti-moat. It actively fills in and paves over whatever moat you might have for price conscious customers.
I think we share the same view here, which I think is most widely discussed as "AI capability overhang". Like you also said, a large, growing amount (digital) tasks can already be completed reliably with existing models. For those - this was my point - the models are competing on price, speed, reliability, etc.
I also agree that more capable models are still desirable in plenty of spaces. I just see that there is so much lag in the diffusion of existing models, which, again, are already sufficiently capable.
When it solves your problem. I don’t think it makes sense to globally qualify a model as “good enough” any more than it does with programming languages. Python is good enough for a web app, but not for an OS kernel.
To your point, I think we’re past the hump on the Pareto distribution. The models are good enough for most things, and the rate they’re improving at the things they’re bad at is dropping. The odds a new model is good at something the old one was bad it is okay, the odds it’s something you care about is dropping. It makes a lot of sense to discriminate on price rather than features in that environment.
"640K ought to be enough for anybody"
I know this gets a lot of flak because it ended up being obviously incorrect for consumer markets.
But the under-reported corellary of this is that a TON of industries essentially operate on hardware right out of that time period (or slightly after, circa 2000s)
banking
Insurance
Healthcare
Gov
retail
Transportation
etc...
Hell, I was writing code to run on IBMs Z/arch as recently as 2021 for banks with mainframes deployed around 2004. They were just then finally considering an upgrade to the later models.
The Chinese labs are just arms of the Chinese government. They have no obligation to be profitable or please (private) investors. If they weaken/collapse American models, then that is the ROI to the lab owners.
This is a pretty strange take on an article explicitly pointing out the profitability of chinese models.
Further, even if this is your take (and I won't touch the geopolitics here, maybe you're right, maybe you're not regarding ulterior motives) - go use non chinese open models that are also good.
Laguna, Nemotron, Mistral, Arcee, Llama, etc...
China has sandbox capitalism. In the sandbox it looks and feels like capitalism, but the sandbox itself exists in a communist framework, and the state still controls all the inputs, outputs, and can intervene at any point as they please.
In organic capitalism you need to compel arrays of discrete private entities to invest in your idea. In sandbox capitalism, the money can just appear in any amount at the behest of a single controlling individual.
Hyperbole and trappings aside, that describes all mixed-market economies. And all major states have champion industries.
Now do American Capitalism!
You can see it plainly on the Solar PV market. The likes of Trina, JA, Jinko, etc. all compete aggressively, leading to everyone running at a loss in order to corner the market, and then after a few years the government steps in and orders all of them to raise prices. Happened again at the beginning of this year, raising cost of PV Modules 30-40% for oversees consumers.
Contrast that to US being happy with everyone eating each other alive until all but one company is left standing and can now dictate the prices and enshittify the product to pay back their VC loans.
Isn’t “sandbox capitalism” - real capitalism and what the us is trending towards is the “Wild West” capitalism with monopolies, trusts, and all the other anti competitive things that arise in the free market if there is no external force that tries to keep the market “fair”?
I mean this is not the case for either, but the systems are setup to drift in these directions - a market needs some sort of regulation to keep things sane, otherwise you end up with standard oil and the like.
You want the people with guns controlling things to be at least nominally interested in “the people” and produce more than just “return on investment”.
Not saying that China is like this or that US is bad, but saying that China is bad because it is “controlled capitalism” is kinda missing the point of what it should be in the best case.
I don't think its that hard to shoehorn current US into those sentences too, maybe switch word 'communism' with 'kleptocracy'.
Less intervening and micro management but oh boy say something not nice about government and potus and suddenly you are a new enemy of the state and 3-letter agencies work day and night to bring you down for good. Or in reverse - put a public bribe to potus and his clan and enjoy some favors for few weeks.
2 years ago I would agree more strongly with you, but we are in new era from now on.
to be fair they did say 'organic capitalism' - which I interpret the same way as 'true communism' in that it probably cannot really exist.
real-life, unregulated capitalism lends itself to exactly what you described
real-life, unregulated capitalism lends itself to exactly what you described
In actual capitalism, POTUS wouldn't have enough power to be worth bribing.
The problem isn't the money, the problem is the power.
nice
> I think everyone has known that inference is profitable overall for the major providers
> The major providers are unprofitable after you add in the training and research costs and other overheads
It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
The profitabiity (or not) of AI models needs to be calculated on a per model lifetime basis - what did the model cost to develop and train, what is the lifetime revenue the model generates, and what is the lifetime operational cost of the model (including amortized GPU cost), etc.
It makes no more sense to calculate "inference" profitability when ignoring training cost than it would to calculate it when ignoring GPU cost.
Exactly. And you can't just stop training models, because your competitors won't, and customers are switching providers on a daily basis (most likely an exaggeration).
Hence "we need to slow down progress for safety" [narrator: it had nothing to do with safety]. The industry is begging for the regulation so that there is a coordinated slow down and consequently move towards profitability.
> coordinated slow down and consequently move towards profitability
100%
Amodei has been promising the "end of the exponential" all year, when they can shift some compute away from cash-burning research and training toward cash-generating inference.
"Pacing the frontier" is how you collude with your competitors to all decelerate at the same time because safety.
I saw last week that Anthropic is currently cash-flow positive with 75% of compute going to research and training. If they can flip a switch and move from 3:1 to 2:1 or 1:1 training:inference, they will be insanely profitable.
As long as they can avoid a price war.
>> calculated on a per model lifetime basis
True only if you never decelerate R&D.
But "pacing the frontier" means every early model doesn't need its own profitable P&L.
Early models can be big money losers while helping build the next generation.
If the industry can coordinate to all decelerate at the same time, then they'll flip from incinerating billions to minting billions, and all that capex will suddenly look completely sustainable.
Unless DeepSeek suddenly gives it all away for free, which is why the big three need government regulation to prevent foreign models from busting the cartel.
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
At the same time, training costs don’t scale with usage, so no matter how expensive they are, they can be paid for as long as inference is profitable. It is only inherently unsustainable if you lose money on inference.
Of course, it could be that the inference usage required to pay for the training is too high to be possible, but that is a separate argument.
It doesn't just matter that inference is profitable - cash from profitable inference needs to outweigh cash burned by training. If training costs even $1 more than your profits from inference on a model, you lost money on the whole thing.
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
Don't enter that rabbit hole, you'd be surprised at how questionable are some accounting rules and practices, even in publicly traded companies.
if i can run an existing open model because its good enough for what most people need, I can just spin up an inference business, and put my spend either on trying to optimize it further or on customer experience
... and not worry about training cost at all.
a lot of the cloud businesses of the last decades have relied heavily on open source code, again skipping most of the cost to make the product/service being sold.
>I think everyone has known that inference is profitable overall for the major providers for a while
Have you seen any place that is not as tech-savy as HN? e.g. some Reddit subreddits?
Barely anyone is aware other than people who are already somewhat deep into this whole topic.
The insider baseball in AI is extreme I agree. I still know people who insist large language models are just glorified autocomplete.
Frontier models are still transformer based, so that's still true.
I believe they're referring to the user experience, lots of people think you just use AI to finish a sentence a la early Github copilot. The average person hasn't seen what agentic flows can achieve
Yes. I've always suspected inference is profitable.
But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?
For most cases it's probably ok, often times answers are general or basically a web search anyways, but if my Agent started blasting outdated syntax it would suddenly be way worse for coding for example.
It’s helpful for a model to be somewhat static when it’s involved in business processes.
Relying on an LLM's baked-in syntax knowledge is suboptimal, compared to leveraging reasonable context affordances (skills, examples, tool calls, MCP servers, search) to ensure adherence to precise APIs and language specs. Don't get me wrong; a model specifically trained on your desired framework is obviously preferable for more consistent generation of idiomatic code -- but relying on that in a large general-purpose model is a questionable proposition IMHO.
They need to be retrained often is my experience for general usage.
I have SO many cases, for example when shopping, when they make assumptions based on their cutoff date, and they need to redo everything based on lots of searches.
Sure, for many use cases old models will be useful but for general usage a model that's 5 years old will probably be garbage for most people, unless they prompt it extensively, which the general public won't do.
> But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?
Just wait until what Ilya Sutskever has for Safe Super Intelligence (SSI) to self update the weights without retraining them to bring training costs down close to $0.
> I think everyone has known that inference is profitable overall for the major providers for a while
A lot of AI haters like Ed Zitron and their readers have refused to update on this. I don't know if any of them have yet, but it'll take a year or two for the idea to fully filter out of their podcasts anyway.
Ed Zitron is a professional apocalypse predictor. Which is to say he's always wrong.
Seems a bit of a disingenuous framing. Most of the arguments I see are some mix of
1. Their figures have been pretty dodgy or excluded training costs so why is it trustworthy this time?
2. API inference charged at a profitable level rather than heavily subsidised subscriptions means everyone tightens up their usage + governance Which we very much saw with the rise of openrouter et al.
Personally I think there's a lot of the bait and switch of 2 when people talk about profitability of the 10-20% of API usage for Anthropic while also wanting to lump in subsidised chat users as if they're profitable too.
I think we know if it was this clear cut then the behaviours of companies would be wildly different and more consistent with profitability.
There are thoughtful versions of these arguments, sure. There are also thoughtless versions, and it's not disingenuous to note it.
It's not actually clear that subscription users are unprofitable as a group (and the group level is the only important one).
We do know the companies are not currently profitable, because they're open and intentional about that, but that tells us nothing about their margins without infrastructure buildout and investment in training.
It’s not training and research that kills you here, it’s your own investors ROI expectations that murder you cold when faced with Open Source commoditizing you, from a country that spent 5% of the sum you spent to get to the same position and then gives the recipe away to everyone.
> I think everyone has known that inference is profitable overall for the major providers for a while
Unfortunately while in the tech industry some might know that, the general public are still being misled (especially by certain folks who’ve made a living off downplaying AI) to believe inference itself is unprofitable.
> The major providers are unprofitable after you add in the training and research costs and other overheads.
This is correct for now, but I recently read that 90% of Anthropic's code is written by Claude Code itself. So, if an AI is using its previous version to train the next version, we're not far from the day where these models will take up the decision making process too. For eg: Decisions API by OpenAI or Jev is based exactly on this. What if a model like Jev sits behind Opus 5.5 and guides it to take the next step? Wooh.
Probabilistic predictions right now are at 99% for frontier models. They will never be 100%, but if humans rate themselves 99.9% in decision making, that benchmark can be broken.
As far as I know they haven't been doing any mass layoffs.
So if Claude Code is writing 90% of the code, then why are they keeping probably the most expensive engineers in the world on the payroll?
The machine writing all the code seems to not be enough, even for the company with access to the best models. What's the alternative to explain this discrepancy?
Frequently it's to keep competitors from having them.
In this game you "win" only if you arrive first, so keeping the same headcount to run faster is a valid option for Anthropic. In this context "winning" means mainly that we lose our jobs
Honestly that's an interesting statement because in my experience Claude writes around 5-10x more code than a reasonable person would to accomplish the same thing. If you include comments then that number goes way up as well.
As someone who has long bought into the idea that the labs are not profitable on inference, I have a question. How is it simultaneously true that there is more demand for inference than supply (apparently by orders of magnitude), but also that inference is so cheap that labs can make healthy margins? It seems like the inference providers, not the model providers, should have the leverage. Why haven’t they said, “give me a bigger cut of each token or I won’t serve your model”?
Also, what happens when the inference providers suffocate on their own debt burdens? Who will serve inference?
It's a few things I assume:
1. Inference providers are largely a commodity business. While demand outstrips supply, there are many providers of that supply, which does somewhat constrain how much leverage a provider has.
2. The models are not interchangeable as of yet (though this seems to be changing). A lot of people want the latest Anthropic or OpenAI model; as such, the labs still have considerable leverage.
3. A lot of the inference providers have used their leverage by doing equity deals with the labs, agreements to have some of the deal revenue spent on compute, etc.
What's the evidence that demand outstrips supply? Is there a wait to get an inference provider or interruptions in availability?
GPU prices are sitting well above MSRP, even for older generation cards. Datacenter growth is also being pushed pretty hard by most of the cloud providers. More anecdotally, the model providers I use have been "throttling" my requests (usually by bumping them down to a lower model version) more frequently. Granted, that could be for profit reasons, but I think it all generally points to a lack of capacity.
>Is there a wait to get an inference provider or interruptions in availability?
I think that the inference providers and model vendors are mostly handling it behind the scenes with more aggressive quantization, increasing API prices, deprecating more generous monthly tiers, etc.
this info is dated
Yeah I think one of the less considered costs when people talk about this is the human cost of training. We all talk about the billions spent on compute, but I don't think anyone's got good numbers on how much RLHF is costing but it's got to be pretty large.
The only moat is in datacenters. The software has no moat, since 1) it's already trained on stolen data, and 2) someone with access to AI will be able to reverse engineer a proprietary implementation eventually, the same AI is being used for developing AI.
That's why so many talks about regulations. Regulations means: ban outside competition, just like US bans Chinese EVs today.
regulation is braindead
I would classify training and research more as capital investment than overhead.
Most people in my experience refuse to believe it for some reason.
Well, as that means there's lots of money to be made lowering costs for research and training! What tools have we used in the past to lower those costs? Automation.. Hm, what if we had AI completely automate AI research? Uh oh, we just rationally found outselves in a trajectory where we lose control of our own trajectory because of economic incentives.
It is wild how fast we're hurtling towards this outcome, lmao.
I doubt that, I got downvoted several times pointing out inferencing is insanely profitable.
Even take account of R&D cost it is still profitable, because you do not actual retrain the base model every month, only distill and post train from the same foundation.
heck, I think even including the giant cost of the top tier researcher salaries, they are still profitable.
then, how are they still losing money? just read what they are actually spend the most budget: always the hardware. those companies will never profitable, because they will never stop spending money to expand their computational prowess.
I don't think so. The number of comments I see from people talking about AI, there are plenty who still think the price is going to shoot up. Some even talk this way about models you can run locally on high end yet still consumer level hardware.
I think the more someone works with AI the more likely they realize that inference does not have to be sold at a loss, but many people's AI knowledge seems quite dated and it sounds almost like it becomes a convenient conspiracy theory for some, a way to convince themselves that the market changes that are already happening a temporary (and I don't mean in a Jevon's paradox sort of way, but that companies will go away from AI).
I suspect a lot of this is fueled by a few stories of companies who went all in on spending as much as possible on AI and then begun to walk that back, without realizing that was more about bad incentives and over promises of what AI could do (especially 1/2 model generations ago). Some also confuse running the SotA models at the fullest capacity costing more and more with a given problem costing more and more to solve, not realizing that the cost per problems is trending down but newer and larger problems can be solved for even more, so the costs at the top are rising despite falling prices. Also things like the subscriptions getting more expensive, though how much of that is because developers are doing far more than they were 6 or 12 months ago (and in some cases there was subsidization that is going away). And maybe a bit with the free models starting to not be upgraded or the harnesses getting worse performance of lowered free limits.
If the frontier labs are struggling under the training costs and the leading open source models are making their progress by distilling from the frontier labs it sounds a bit like there’s something wrong with the economics here or that progress is going to slow quite a bit without the amount of investment.