Besides generating the tests, automatically running tests on edit and showing the results inline is already a thing. I think it'd be better to do it the other way around, start with the tests and let the LLM implement it until all tests are green. Test driven development.