I do something similar. I've recently started having a few "starting point" files to re-explain common context (less than thirty lines per markdown file, usually) that I can point the agent at at the start of a new session, each tightly scoped to a certain domain and/or task type. That's been nice to avoid repeating myself, without the side-tracking or over-aggressive biasing-towards-previous-conversations that I've seen happen if I use long sessions or let it try to decide on its own what to pull in from larger files or trees of files. Sometimes I'll tell it to update that file based on new info from a current task, but I keep tight control over what gets pulled into task start context.

They aren't really "explaining the project" either, but more module- or task-specific preferences, hand reference pointers, or other things like "there are mixed examples of how to do certain things in this project, prefer X to Y." I use a write-everything-twice approach. After I find myself having to correct an implementation because it didn't figure out one of these things on its own from the existing code, I'll add an entry. That also avoids bloating things with "I think this is relevant" compared to "I have noticed that this is necessary."

I keep doing this because it lets me experiment with different approaches to problems without risk of it fixating on things from a previous abandoned attempt, and particularly because sometimes I'm wrong and I haven't found the agent harnesses particularly reliable at taking my word for it from a POV of "yes I know I said we need xyz earlier, but let's please entirely forget about that."