Yes this ^
It totally is a pipe dream to find some magic flat human readable file that does all this, you need to think of it as a config system not a config file.
So then why aren't we questioning our core assumption here that the solution has to be the shape of a human readable file format at all and accepting "syntactically optimal" as the best we can come up with?
Other than that text files are a compatible format to work with for the tools that coders use to interact with everything (text editor/git) why does it need to be a text file at all? or even human readable? Should we even call csv human readable... ?
The perfect config system you describe could equally be implemented on top of any kind of storage (file, db, whatever) with a configuration editor ui to drive it all. And thanks to llm's even for totally one-off custom configuration files making a small web editor for them is basically free if you can give it the schema and some examples.
And ui doesn't mean it can't be keyboard driven (just to preempt people griping about using a mouse). But it does mean that you can present your config in the way that makes the most sense to actually do the configuration instead of having to work around any limitations of a syntax. An example is the configs out there with the trailing comment "foo bar #foo bar sets the the foo bars the bar in the blaze." Maybe you've configured foo 100 times, you don't need the comment to know what foo sets but the syntax demands it so you have to see it every time. In a dedicated visual config editor the comment can be pulled up only when needed.
Will your config "file" end up with some extra cruft to store things about how to show it properly in the ui? totally. but we don't care because we don't actually edit that "file" directly anymore remember?
I do think human-readability is a useful feature, to some extent. Human operators should be able to sanity-check individual components of the system for correctness, and this is so much easier if intermediate data formats are "human readable" (which I would extend as far as .sqlite, assuming a straightforward schema design).
Yeah I’m not saying you shouldn’t have a file that is possible to sanity check if that’s important to your system, but I would say (contrary to op and many comments) that we shouldn’t design around that as the core feature assumption. We don’t have trouble sanity checking our normal db tables so I’d imagine that any format that has a reliable way to see the loaded-in-memory representation would meet the same level of sanity checking.