I do as well, with Codex though, but OP is asking for more fine grained control of what's in context and what can be thrown away.

You can simulate this of course by doing the reverse and maintaining explicit memory via a markdown files or w/e of what you want to keep in context. I could see wanting both, since a lot of the time it would be easier to just say "forget that last exploration we did" while still having it remember everything from before that. Think of it like an exploratory twig on a branch that you don't want to keep.

Ultimately I just adapt by making my tasks smaller, using git branches and committing often, writing plans to markdown, etc.