I have a proposal that addresses long-term memory problems for LLMs when new data arrives continuously (cheaply!). The program involves no code, but two Markdown files.
For retrieval, there is a semantic filesystem that makes it easy for LLMs to search using shell commands.
It is currently a scrappy v1, but it works better than anything I have tried.
Curious for any feedback!
The markdown approach has a real advantage people underestimate: you can read and edit the memory yourself. With vector DBs and embeddings the memory becomes opaque — you can't inspect or correct what the model "knows". Plain files keep the human in the loop.
The hard part is usually knowing what +not+ to write down. Every system I've seen eventually drowns in low-signal entries.
The editability is surely an underrated advantage, both for the program itself and the memories it generated.
I think in terms of noise, it is less problematic here because not everything is being retrieved. The agent can selectively explore subsets of the tree (plus you can edit the exploration policy by yourself).
Since there is no context bloat, it is quite forgivable to just write things down.
This assumes that the model's behavior and memories are faithful to their english/human language representation, and don't stray into (even subtle) "neuralese".
Is there anything (besides plumbing) that prevents both? i.e. when the file is edited, all the representations are updated
[dead]
I've seen a lot of such systems come and go. One of my friends is working on probably the best (VC-funded) memory system right now.
The problem always is that when there are too many memories, the context gets overloaded and the AI starts ignoring the system prompt.
Definitely not a solved problem, and there need to be benchmarks to evaluate these solutions. Benchmarks themselves can be easily gamed and not universally applicable.
The armchair ML engineer in me says our current context management approach is the issue. With a proper memory management system wired up to it’s own LLM-driven orchestrator, memories should be pulled in and pushed out between prompts, and ideally, in the middle of a “thinking” cycle. You can enhance this to be performant using vector databases and such but the core principle remains the same and is oft repeated by parents across the world: “Clean up your toys before you pull a new one out!”
Also since I thought for another 30 seconds, the “too many memories!” Problem imo is the same problem as context management and compaction and requires the same approach: more AI telling AI what AI should be thinking about. De-rank “memories” in the context manager as irrelevant and don’t pass them to the outer context. If a memory is de-ranked often and not used enough it gets purged.
Fair concern.
ReadMe does support loading memories mid-reasoning! It is simply an agent reading files.
Although GPT-5.4 currently likes to explore a lot upfront, and only then responds. But that is more of a model behaviour (adjustable through prompting) rather than an architectural limitation.
Ah, I mean bi-directional management of context. Add and remove. Basically just the remove bit since we have adding down.
Mid thinking cycle seems dangerous as it will probably kill caching.
The mid thinking cycle would require significant architecture change to current state of art and imo is a key blocker to AGI
Context bloat is real, but the architecture has the potential to solve it.
You need clever naming for the filesystem and exploration policy in AGENTS.md. (not trivial!)
The benchmark is definitely the core bottleneck. I don't know any good benchmark for this, probably an open research question in itself.
What is the memory system you are referring to? I've been trying Memori with OpenClaw. Haven't had a ton of time to really kick the tires on it, so the jury's still out.
I love how you approached this with markdown !
I guess the markdown approach really has a advantage over others.
PS : Something I built on markdown : https://voiden.md/
Yep. Markdown is the future :-)
I really like the simplicity of this! What's retrieval performance and speed like?
Minimalism is my design philosophy :-)
Good question. Since it is just an LLM reading files, it depends entirely on how fast it can call tools, so it depends on the token/s of the model.
Haven't done a formal benchmark, but from the vibes, it feels like a few seconds for GPT-5.4-high per query.
There is an implicit "caching" mechanism, so the more you use it, the smoother it will feel.