I'm not sure it's the context window. Gemini cli keeps a shadow git repo to be able to rollback changes in cases like this: https://gemini-cli.xyz/docs/en/checkpointing.

It's disabled by default, but even with the default setups, you can find large snippets of code in ~/.gemini/tmp.

tl;dr: Gemini cli saves a lot of data outside the context window that enables rollback.

I'm sure other agents do the same, I only happen to know about Gemini because I've looked at the source code and was thinking of designing my own version of the shadow repo before I realized it already existed.