>The tool-call history collapse is a problem I'd pay real money to have solved cleanly.

It's general attention collapse and it happens everywhere once you start noticing it.

The simplest example, which even frontier models fail at, is something of the form `A and not B', which they keep insisting means `A and B' after the text gets pushed far enough back in the context.

The only solution, I think, that is even theoretically capable of fixing this is using a different form of attention. One which innately understands tree-like structures and binds tree nodes close together regardless of overall distance from the end of the stream.

Incidentally this is what I'm also working on at $job.