I have basically polluted my context and agent files, agent.md, claude.md, gemini.md with really strong guidance about using plain language. I even include small, mini design specifications, examples of plain engineering language, only using technical language and invented terms when they actually make sense (almost never). EVERY TURN.
I have a pretty large set of prompts that go into any software engineering, and I force every single agent to use an ephemeral style stack of prompt management. So, every turn it goes to the top of the stack and it is the very last thing they see in terms of all of my prompts and instructions and agent files. And then it gets taken out of the conversation so that it doesn't get sent to the agent the next turn (no context bloat). It has restored so much sanity.
I tried the caveman add-ons, and I felt like I was losing IQ points because I spend a lot of time reading agent output, and when they start talking like cavemen, I start thinking like cavemen. That was not good for my mental health. So, I try and make the agent talk like me and think like me. And it works, mostly. And my observation is that maybe I'm not the most efficient agentic thought process, but my sanity is retained.
All of that is to say that if something is reading like that to you, just have the agent rewrite it and read it in a rewritten tone because it's probably bad as it stands and your colleague did not put enough effort in it. It is /not/ good and you should not accept it as a default. We have to hold the line on stuff like this and maintain some semblence of normal human engineering standards that existed before AI. They are not making us better. They are making is lazy and dumber.
Opus 5 and other agents in the latest rounds of tuning have gotten ridiculously bad in terms of how they feel to interact with with all the invented language and localized nomenclature. It is an obvious bias that big words and technical talk looks good to the bottom of the bell curve, but when you actually try and understand it, it's horrible. So people say, "Yeah, that looks great," in all the RLHF rounds, and they run with it because they think it looks good, but it doesn't. It's terrible.
Hold the line. It isn't you. And it isn't a good methodology document.
"nd then it gets taken out of the conversation so that it doesn't get sent to the agent the next turn (no context bloat). "
Uh - dude - this means you're paying 10x in token costs because there's no caching.
If you 're-write token history' then you can't cache tokens.
It means for any reasonably long conversation, the llm has to reprocess the entire history as preflow on every prompt.
Are you sure you're really doing what you say you're dong, and how is it not blowing up your budget?
Nope, you gotta understand. This is the move. It is appended at the end of every single message. It is never saved in the conversation that get sent back for inference. So you send it. But when you go back for inference, it’s at the top of the stack so all of the cashing works you’re not pre-filling every time or anything like that. It burns plus N tokens, where N is my prompt stack. It is not really that expensive. I have measured it to within an inch of it its life. Think of it this way every bit of the prefix and the conversation stays exactly the same you’re only adding to the very end of the conversation. So after the first turn, it is basically always cashing within the KV cache for a given context. Sorry I am using voice dictation. My hands are tired this week. Basically you sculpt the conversation history to ensure prefix caching
Yes - with arbitrary prefix caching that might work. That's tricky though, not everyone is going to provide that.
Did you have to build your own harness for this? Or hack Claude Code or something?
I don’t use their CLIs. OMP/OpenCode. If you are stuck on them, you can do it in a proxy layer.