> No, you don't. Without training cost you can infer only the marginal cost of serving this kind of models.
Still useful; "are the labs marginally profitable just on the marginal inference costs?" is still a useful question to answer. After all, if they aren't even profitable on inference in isolation, then we can expect to see large price increases.
If they are able to turn a marginal profit on inference alone, then perhaps the price increases won't be so severe (or perhaps they expand the time between generations so that they spend less on training but take longer to complete training).
"Are the labs profitable at all?" is, of course, a much more useful question, but that doesn't mean that the first question is completely useless.
I wasn't saying it isn't useful, I was saying that you cannot infer even the marginal cost of closed models.
The K3 maths can turn true only if the models size is roughly the same and the labs didn't find any better way to run inference at scale.
We know labs make money on inference, and we know they lose a lot of money on inference+training.
> We know labs make money on inference
We don't really know that, for OpenAI and Anthropic. We suspect that, but as far as I know, even they have stopped claiming that they are profitable on inference.
unless you think that Opus is 10T+ params, its pretty much impossible for inference not to be profitable when doing some basic napkin math on other open models, and if Kimi K3 is 3T params with the same performance as Opus then that means that China is actually way more technologically advanced than the American labs.
So which is it?
Just out of curiosity, based on what we know for sure they(OAI+A) make money on pure inference and lose on inference+training?
OpenAI's financials leaked and showed this pretty convincingly.
Anthropic was probably profitable last quarter, without training costs: https://www.wsj.com/tech/ai/mind-blowing-growth-is-about-to-...