Also, the cache hit pricing is 25% of the input pricing ($2 vs $0.50). Long agentic workflows are dominated by cached input. The US frontier labs typically have this at 10% of the input price, and DeepSeek/Xiaomi etc take it to the extreme 1% range (which is why those are cheap to run in real world agentic loops with dozens of toolcalls per run)
Just to add context (no pun intended), OpenAI also charge differently based on context usage with GPT 5.5 being $5/30 below 200K, and $10/45 above.
Anthropic have a fixed price regardless of context usage.
These per-token pricing schemes aren't directly comparable though since these models all use different numbers of tokens, even for input (Anthropic's recent tokenizer change generates 30% more tokens for exact same input), as well as for reasoning, and context/token usage also varies wildly by harness with Claude Code using 3x the context/tokens of Pi.
Think of the entirety of the context (the full thread of conversation, all tool call output, etc.) as one message that's been submitted to the LLM all at once just so it can generate the next token (which is just the next word or even syllable.) Once that token is generated it is appended to the context, and the entire context is once again used to generate the next token. Keep doing that until the entire response is generated. The larger the context, the more stuff the model has to pay attention to as it generates each next token.
To use an analogy: imagine your friend is the author of an unfinished book. They die with 19 chapters written, and on their death bed ask you to write the 20th chapter. Assuming you're up to the task, you can only do this well if you take the time to absorb the entirety of what's been written so far.
This is how LLMs work. Context caching is an optimization on top of this, but it has its limits.
I guess it does increase their cost, or rather your share of their hardware depreciation.
AI serving cost is apparently mostly hardware depreciation rather than operating cost (electricity etc), and if your large context request is occupying VRAM for some fraction of a second then you are paying for the depreciation that occurs in that time!
e. H100 costs $20-40K to buy, with a lifetime of maybe 3 years, and will only consume maybe $2K in electricity if run 24x7 for those 3 years.
Yeah but depends how you use it - with superpowers and it’s prevalence of splitting things into smaller focused subagents - this could seriously reduce costs …
I wish my company gave me more options than just using Claude to test these things out
The recent Databricks comparison has GLM 5.2 performing identically to Opus 4.8 on high effort, and some early Twitter reports (e.g. from the OpenCode developers) strongly favor GPT 5.6 Sol over Fable.
As always it depends on what you are using them for, and how you are using them.
Also, the cache hit pricing is 25% of the input pricing ($2 vs $0.50). Long agentic workflows are dominated by cached input. The US frontier labs typically have this at 10% of the input price, and DeepSeek/Xiaomi etc take it to the extreme 1% range (which is why those are cheap to run in real world agentic loops with dozens of toolcalls per run)
Just to add context (no pun intended), OpenAI also charge differently based on context usage with GPT 5.5 being $5/30 below 200K, and $10/45 above.
Anthropic have a fixed price regardless of context usage.
These per-token pricing schemes aren't directly comparable though since these models all use different numbers of tokens, even for input (Anthropic's recent tokenizer change generates 30% more tokens for exact same input), as well as for reasoning, and context/token usage also varies wildly by harness with Claude Code using 3x the context/tokens of Pi.
I didn't know that, thank you.
Does anyone know why they would charge more for higher context usage (other than they can)?
Think of the entirety of the context (the full thread of conversation, all tool call output, etc.) as one message that's been submitted to the LLM all at once just so it can generate the next token (which is just the next word or even syllable.) Once that token is generated it is appended to the context, and the entire context is once again used to generate the next token. Keep doing that until the entire response is generated. The larger the context, the more stuff the model has to pay attention to as it generates each next token.
To use an analogy: imagine your friend is the author of an unfinished book. They die with 19 chapters written, and on their death bed ask you to write the 20th chapter. Assuming you're up to the task, you can only do this well if you take the time to absorb the entirety of what's been written so far.
This is how LLMs work. Context caching is an optimization on top of this, but it has its limits.
I guess it does increase their cost, or rather your share of their hardware depreciation.
AI serving cost is apparently mostly hardware depreciation rather than operating cost (electricity etc), and if your large context request is occupying VRAM for some fraction of a second then you are paying for the depreciation that occurs in that time!
e. H100 costs $20-40K to buy, with a lifetime of maybe 3 years, and will only consume maybe $2K in electricity if run 24x7 for those 3 years.
Womp. Didn't see this anywhere else.
No longer feels as inexpensive. Will likely just include this in the rolodex of <200k context tasks, like being one of my review agents.
Yeah but depends how you use it - with superpowers and it’s prevalence of splitting things into smaller focused subagents - this could seriously reduce costs …
I wish my company gave me more options than just using Claude to test these things out
Claude Fable and Opus 4.8 1M are by far the best, smartest models. Anything else is a downgrade so you’re not missing anything.
The recent Databricks comparison has GLM 5.2 performing identically to Opus 4.8 on high effort, and some early Twitter reports (e.g. from the OpenCode developers) strongly favor GPT 5.6 Sol over Fable.
As always it depends on what you are using them for, and how you are using them.
That's very notable and left out of the announcement.