Why do the cache hit rates seem to vary so much between harnesses?
I use pi, which is very minimalist, and I get a hit rate of ~99%. Paying like $1 a day for Flash. Yet, the hit rate mentioned on OpenRouter is only ~79%.
Yes hit rate does vary by harness and by how you use the harness. If you use subagents, for instance, they will start with a whole new context created by the main agent, and this will not be cached. If you mostly use the main agent with Pi, you’ll have high hit rates and low costs. Sometimes agents do “cache busting” things where they’ll move around some of the text in the context to try to keep old instructions from being forgotten, thus keeping the agent on task, and this will bust the cache. I’ve heard, but not validated myself, that Open Code has some issues with this.
BTW, this is one of the things that I really like about Pi. It’s very simple and thus very predictable.
Half OT:
Why do the cache hit rates seem to vary so much between harnesses?
I use pi, which is very minimalist, and I get a hit rate of ~99%. Paying like $1 a day for Flash. Yet, the hit rate mentioned on OpenRouter is only ~79%.
Yes hit rate does vary by harness and by how you use the harness. If you use subagents, for instance, they will start with a whole new context created by the main agent, and this will not be cached. If you mostly use the main agent with Pi, you’ll have high hit rates and low costs. Sometimes agents do “cache busting” things where they’ll move around some of the text in the context to try to keep old instructions from being forgotten, thus keeping the agent on task, and this will bust the cache. I’ve heard, but not validated myself, that Open Code has some issues with this.
BTW, this is one of the things that I really like about Pi. It’s very simple and thus very predictable.