We have an internal proxy (that I've been meaning to open source for ages) that routes all llm usage at our company, which allows us to see data in realtime. Its been fascinating how rapidly Pi has been adopted. Moreover since its pretty hackable, we've been able to automatically aggregate context from pi sessions, which has resulted in Pi efficacy being higher as more people use it, putting in place a interesting virtuous loop. I didn't expect this outcome: for whatever reason I assumed proprietary harnesses fine tuned to work with a companies' models would work better? ps/random aside: there is something slightly off about Pi's edit command, we are planning to investigate this further and patch this as we have quite a few session traces now..
Yes, this is a known issue. A significant amount of Edit tool calls fails in Pi witg newer models.
https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/
https://blog.can.ac/2026/02/12/the-harness-problem/
It's worse than I expected.
I don't understand peoples expectation. If our language skills were as non-ambiguous as a coding language, we'd have solved world hunger by now.
So why would we expect all these bizarre bizantine language models to all conform to how a request is both made, expected and massaged.
For awhile, I was getting bizarre opencode tool errors where the only problem was the model was passing in a "1.0" or "0.0" where the harness dutifully wanted an integer. Of course 0.0 is the same as an integer in practical operations.
I have one core complex task where there are a number of simple errors like this. The easiest thing for me was to just have a post-processing script that performs: lint > mark known fail-early results > fix common errors (all formatting or type errors right now) > relint > final pass / fail > pass to next step (retry with tweaks / user in the loop since I don't have the token budget to just blindly keep trying the same thing).
[dead]
[dead]
I keep hesitating to use Pi because I primarily use Claude and I worry it will be much more expensive to pay for API usage rather than the bundled subscription usage. But I haven't actually evaluated the cost tradeoff. Should I get over this hesitance and dive into Pi?
Aside 2: Anecdotally we found that Pi performs more or less on par with native harnesses at lower cost on decently specified prompts. It is also phenomenal at context cacheing especially on Deepseek models (its hard to precisely attribute credit here are my understanding is this is a DS speciality). But it fails much worse on poorly drafted prompts. I'm generalising but native harnesses seem to be better kind of flailing along on those.
Is it because Pi’s default system prompt is so simple?
of course it is; these entire systems are about priming the LLM to perform X task; if it's a coding harness, giving it very little priming for a task is going to make it grasp at straws.
Can anybody share a tested system prompt they use for general coding tasks in pi?
clone opencode and ask your ai yo find it. i did that then just wrote my own primary open code agent.
> we've been able to automatically aggregate context from pi sessions
what do you mean by this ? do you rewrite the context in your proxy ?
I imagine they're capturing all the tokens+context from pi sessions from every developer.
Yeah.
Can you say more? This sounds like it could be more useful than trying to build a knowledge base (say qmd) and then get the models to search it for the right context each time.
This is in fact what we do (with higher order abstractions now built on top of this). This builds self evolving interactive knowledge base and puts it into a QMD searchable index. The indexer is already open source: https://github.com/jibs/duffel
Can you explain context aggregation here more? What is that? What are you doing and how is it helping?