I'm actually building this and believe I've overcome the most difficult aspects mentioned here. It will be released as Open Source next week. https://intentcode.dev