I think the central question for a such memory system is whether we or the agents can find the relevant information and how to organize these data as changes continues to come in. Would we miss something in the retrieval process? How do we organize the information so they stay actual and correct without piling up the garbage? Of course we can continue to concatenate the data and tag them with version and date, but then we have to face the problem of extracting the relevant information in a short time. If we delegate that problem to a LLM, long context retrieval performance will degrade and the cost will explode.
That is the reason why we condense the information in the first place. Forgetting + Synthesizing are the necessary parts of learning and basically with memory + smart retrieval we want to build a learning system.
What I'm building has a regular maintenance it runs across the knowledge. A. to chunk long articles down into smaller chunks or information with good routing. B. To review the latest thinking to make sure old thinking is deprecated and linked.
This will be the big challenge with this. Storing/Retrieving works fine across hundreds of articles... I have more than 500 right now just from OzBrain and a bit of the thing I was building before. When it becomes thousands with dozens being added all the time, the maintenance will become more of a necessity.
I lean a little bit towards store everything because the system's abilities are changing so fast right now, that having our previous thinking or reasoning is useful in being able to improve or correct current assumptions based on previous data.
[flagged]