Why? Well it depends, most evidence is suggesting that Anthropic and OpenAI are making a lot of money on inference so the question is whether its more profitable for them to sell 100X tokens for Y, or 1X tokens for 100Y. In most industries with high fixed costs and low variable costs and unlimited scalability (like LLM providers) the first option ends up being much more profitable
Literally nobody is making money on inference
Based on what? There isn’t a lot evidence that’s the case..
Prices on OpenRouter for GLM and other large open models indicate that Anthropic/OpenAI must have pretty high gross margins even if their models are several times more expensive to serve.
It wouldn’t make sense for any provider to host large open models and then loss $10 on every $1 they make since they don’t have infinite VC money or any business model that would justify it.
If they had high margins they wouldn't be issuing senior debt with a 18.5% coupon payment (and failing to fully subscribe it), nor would they need Elon to give them two months of free compute in order to appear profitable for a single quarter.
We were talking specifically about inference and I don’t think there any indication that their gross margins on the API tokens (if not the personal subscriptions) are negative?
Obviously they have R&D and other fixed expenses that make the company itself highly unprofitable but that’s only semi-tangential.
No I mean Anthropic has only claimed a profitable quarter based on xAI giving them two months of free compute, and both Anthropic and OpenAI are counting discounted revenue as actual revenue. They haven't found a way to sell inference for less than it costs them yet, and when they tried earlier this quarter their customers bailed.
Well again.. you are mixing up inference costs and their other mostly fixed expenses (in addition to sales and marketing)
Is there any indication that if they could sell X * N more tokens than now at the same (or even quite a bit lower) price they wouldn’t become profitable as a company?
> They haven't found a way to sell inference for less than it costs them yet
Based on what? I only see evidence to the contrary.