If I repeatedly call an LLM in a loop with a markdown document it can edit, would that make it qualify for you?

If I give an LLM to compact its context window, so the context it carries can evolve iteratively over time as more and more things come in, is that enough?

Compacting the context is really a very, very interesting example here. The "next token predictor" is telling an external tool to change all "previous" tokens. So an LLM + a harness that allows compacting the context is no longer just a token predictor at all!

You don't need continuous learning to get interesting dynamics. You just need feedback loops.