We built a conversational terraform platform at work, you describe what you need for your infra and it produces the terraform code following company standards and rules. What is important for us is the LLM never writes any code. It just extracts a structured spec from the conversation, and a deterministic engine (plain python, no LLM) renders the code from approved modules only. So there is no "code quality" discussion at all, and since the spec is structured and not natural language, you don't need another LLM to verify it, the risk moves to the extraction step which is small and easy to eval. But probably this only works in narrow domains where the output is composable from approved blocks (IaC, config etc), and its not a new idea, just old discipline behind a chat interface. But in that slice the speedup is way beyond 2x.