> It's widely understood that the big players are making profit on inference.
This is most definitely not widely understood. We still don't know yet. There's tons of discussions about people disagreeing on whether it really is profitable. Unless you have proof, don't say "this is widely understood".
You can look at open source models hosted by various companies that have no reason to host them on a loss.
Uber ran their ridesharing at a loss for years. This is a very common way to gain market share.
What market share? We are talking commodity models where the host does not matter at all at OpenRouter etc.
Uber had massive VC investment and a moat. The companies he's referring to likely don't have much VC investment and zero moat.
I recently had Codex working for 80+ hrs non stop (as in literally that was a single running session in response to a single prompt!).
Even at $200 monthly subscription that kind of stuff burns through tokens at a rate where it's very difficult to believe that they are even breaking even, never mind profit.
thats nuts. what was it doing for 80 hrs?
Probably asked what’s the Answer to the Ultimate Question of Life, the Universe, and Everything
The reality is we can’t trust accounting earnings anyway.
We need to see the cash flows.
I don’t have “proof” but the existence of so many providers of free models on OpenRouter strongly suggests inference is running at a profit. There’s no winner-takes-all angle to being a faceless provider there (often the consumer doesn’t know who fulfilled the request), so there’s just no incentive at all for these small provider companies to exist unless inference is profitable under the right conditions.
>but the existence of so many providers of free models on OpenRouter strongly suggests inference is running at a profit
I don't think it suggests a profit, but rather a _hope_ for a _future_ profit, and a commitment to a strategy that may or may not pan out. Capitalism rewards those who are early to the party and commit to their bit.