We've had "natural language"-esque DSLs for a long time. Cucumber is probably among the better known, at least the first that comes to mind.

> Yet, with LLMs, we commit our generated source code, completely throwing away the English language abstraction.

Unless you want to commit your chats, that's very much a bonus. You don't want two different people "compiling" completely different versions of your application, because LLMs aren't deterministic.