It seems like most of these tools are mostly vaporware. Benchmarks done on Headroom and RTK show that neither result in real savings. If it were possible to have such a simple pre-process step why wouldn’t the AI Labs upstream the optimizations themselves? My guess is they mostly don’t work or make the behavior much more confusing for the model. I really think there needs to be some kind of independent benchmark.
Here are other cases demonstrating the exact same issues with these kinds of tools:
https://blog.jetbrains.com/ai/2026/07/rtk-claude-code-token-... https://brandonbarker.me/writing/headroom-fewer-tokens-bigge...
Naively, I think some optimizations would require access to the whole codebase and that would make people nervous (plus incur more cost).
Eg absurd idea, but you could write something that minifies a codebase (by token, rather than byte) and then translates edits back into the expanded code. Probably an insane use of fuse lol. Partially minifying on each tool call sounds like a huge pain with a lot of state to track.
There’s also a lot of common situations where humans prefer solutions that take more tokens because it’s easier for us to read (eg for loop vs map vs list comprehension), which may have some gains.
I strongly suspect there is some form of token compression that works, but I don’t think it will be as simple as “pipe arbitrary text with no context into this tool”.
Jetbrains feels like a place this might come from. “Take this code, parse it to an AST, find the fewest token representation of it” feels like something they’d do, or maybe Astral (specifically in Python land, type checkers feel sort of adjacent as well).
I fully agree. Some of these tautologies, RAGs or RAG adjacent ideas feel like there is real potential. I know in my own testing adding a knowledge graph provided cost saving benefits.
Ultimately, the problem is people aren't actually benchmarking any of these ideas or if they are its via an extremely misleading methodology. For example, headroom violates cache frequently, nullifying any potential savings it could have; but does their benchmarks show that? Nope. [1]
As an aside, I wonder how many days are we away from Codex or Claude taking all the "read" tool calls and re-assembling a full map of your codebase and then offering it as a feature.
[1] https://docs.headroomlabs.ai/docs/benchmarks
I think people don’t do genuine benchmarks because the market forces them to pretend their solution works for anything you can throw AI at. Companies whose valuation is based on them being the RAG/compression/routing/etc company. They can’t admit it only works well in a specific domain because then they’re immediately $300M in the hole.
I have more faith in companies with a more targeted approach. Eg gzip does fine, but video codecs beat compressing raw video by a ton.
> As an aside, I wonder how many days are we away from Codex or Claude
That sounds like SourceGraph but twice as expensive, although it does have “AI” so probably lol
> If it were possible to have such a simple pre-process step why wouldn’t the AI Labs upstream the optimizations themselves?
Not defending these tools, but one reason these might not be upstreamed is because it would negatively impact vendor margins, and they have no incentive to save their users money
Judging by the leaks, OpenAI and Anthropic already train reasoning traces to use fewer tokens (they deliberately omit articles and prepositions, use very short sentences, etc.), even though you pay per token. So it wouldn't make sense to do that if the only incentive was "make them pay for as many tokens as possible per task."
It's more subtle than that. If a user has to wait longer for a solution/pay more, they'll be less satisfied and may switch to a competitor. More unnecessary tokens also means more unnecessary compute. Longer sessions are increasingly more expensive to serve than shorter sessions.
And there's always the Jevons effect: as a resource becomes cheaper, demand often increases, and so does net resource consumption.
So, imho, frontier labs have every incentive to reduce token usage per task (while also making you use AI for more and more tasks in your daily life)
Even JetBrains is now AI blog-slop, how disappointing.
Not all, but definitely on some. I also find it very off-putting.