Not a source but a comparison with a weaker non-SOTA model:
Nvidia's top AI chip Rubin sells in 72-GPU racks for about $3.5–7.8M. A rack running Xiaomi's MiMo V2.6 Pro generates roughly 150–300B tokens a day, worth about $130–260k at Xiaomi's API price. That's a payback of the infrastructure in a few weeks in theory. After a few weeks or a month, the only cost is electricity, and whatever they make after that is pure profit.
OpenAI and Anthropic are practically scamming people with the token prices.
You probably mean they are operating with 80% margins discounting training expenses, which will continue to be pretty high for the foreseeable future.
Source?
Not a source but a comparison with a weaker non-SOTA model:
Nvidia's top AI chip Rubin sells in 72-GPU racks for about $3.5–7.8M. A rack running Xiaomi's MiMo V2.6 Pro generates roughly 150–300B tokens a day, worth about $130–260k at Xiaomi's API price. That's a payback of the infrastructure in a few weeks in theory. After a few weeks or a month, the only cost is electricity, and whatever they make after that is pure profit.
OpenAI and Anthropic are practically scamming people with the token prices.