They already invested in the massive datacentres of GPUs sitting idle. They have fewer users so they can deliver more inference per user - more thinking, larger models.
There are various difficulties with renting GPUs, especially if your setup is very custom.
The competitor would have to port their training systems to your specific network architecture, system design, rdma Vs ethernet vs infiniband Vs nvlink etc.
Getting it running might not be too hard, but getting it running efficiently and making good use of all those flops will require considerable human effort and wall time.
Add that to the fact most frontier labs seem to have a single huge training run - and to my knowledge nobody has figured out how to distribute that training run between data centers effectively.
I'm not saying this as a hypothetical -- I believe xAI is already renting their GPU farm out to frontier labs. Whatever logistical challenges exist, enough of them seem to have been overcome.
So where are these mythical savings coming from? You're saying they have spent more per user therefore can charge each user less or something? I'm not following.
The (optimistic?) take is that xAI is genuinely better at building datacenters at scale than anyone else, and the freedom to use Nat Gas as the primary energy source allows them to have lower marginal costs.
The (pessimistic?) take is that they have loads of idle GPUs and want to get some revenue out of them rather than none. Compare this to OpenAI/Anthropic where every token used by a consumer has to compete with enterprise spenders, and there’s not enough to go around for everyone.
It's also sensible for them to provide a cheap, intelligent model to users if they have capacity, then once they built a user base, tighten the screws. All the other AI providers have done that.
They already invested in the massive datacentres of GPUs sitting idle. They have fewer users so they can deliver more inference per user - more thinking, larger models.
Don't they just rent them out to the frontier AI shops? They're not sitting idle.
There are various difficulties with renting GPUs, especially if your setup is very custom.
The competitor would have to port their training systems to your specific network architecture, system design, rdma Vs ethernet vs infiniband Vs nvlink etc.
Getting it running might not be too hard, but getting it running efficiently and making good use of all those flops will require considerable human effort and wall time.
Add that to the fact most frontier labs seem to have a single huge training run - and to my knowledge nobody has figured out how to distribute that training run between data centers effectively.
I'm not saying this as a hypothetical -- I believe xAI is already renting their GPU farm out to frontier labs. Whatever logistical challenges exist, enough of them seem to have been overcome.
Surely rented GPUs would be used for inference, which runs anywhere and needs to scale much larger than a training run.
They do, yes.
So where are these mythical savings coming from? You're saying they have spent more per user therefore can charge each user less or something? I'm not following.
The (optimistic?) take is that xAI is genuinely better at building datacenters at scale than anyone else, and the freedom to use Nat Gas as the primary energy source allows them to have lower marginal costs.
The (pessimistic?) take is that they have loads of idle GPUs and want to get some revenue out of them rather than none. Compare this to OpenAI/Anthropic where every token used by a consumer has to compete with enterprise spenders, and there’s not enough to go around for everyone.
It's also sensible for them to provide a cheap, intelligent model to users if they have capacity, then once they built a user base, tighten the screws. All the other AI providers have done that.
It’s basically a clearance sale, is the theory.
More like they have a less focus on margins and more on cost recovery.
Definitely. They had insanely low rates on TTS up until a month or two ago ($4.20/1M) for example, which they only recently started increasing.
As their models get more competitive I'm sure prices will catch up.
“We lose money on every rack, but we make up for it in volume!” - Elon Musk, probably