Check out github.com/ralabarge/beigebox -- OSS AI Harness, started as a way to save all of my data but has agentic features, MCP server, point it at any endpoint (or use any front end with it as well, transparent middleware)

So far what I am finding is that you just get the basics working and then use the tool and inference to improve the tool.