I wonder if "the right solution" is a programming language that is fast, concise, trivially easy to run, and outputs some efficient binary format like protobuf.
Programming languages have comments and control flow, multiple popular implementations, and can have nice literals. Lack of Turing completeness is actually not a terribly useful feature if you trust the input (and you should probably just use protobufs or similar for untrusted inputs in that case.)
> Programming languages have . . . control flow
I do not want control flow of any kind in my configuration file. Nor do I want expressions or any kind of evaluation.
Greppability is a must-have feature for me. As is simplicity - I don’t want to have to deal with internalizing interpreter mechanics, rules for precedence, variable scope, etc just to figure out what config values my program is going to be provided with.
Any time I’ve been forced to work with a system which used a general or restricted programming language to express configuration, it’s been a nightmare.
If your config language doesn't have its own control flow then it's going to get a meta layer added on with control flow. Like, what do you do if this service has a different hostname in staging versus prod? Or connects to a different DB, or whatever. Either have one template file and give it a values file when you go to deploy it, or have two fully-written out files... which becomes annoying when you have to make sure you keep them in sync across two or more environments, often leading to a hacked on and poorly implemented templating system anyway (eg, Helm)
If your program requires executing another program to prepare its options, then something has gone wrong in design or architecture.
Configuration is a type of API - and should be designed with the same care and goals including human legibility, orthogonality, “simple things should be simple to do”, surprise avoidance, stability, minimalism, etc.
Instead often what we get is little more than a sort of serialized form of the internal state of the system.
If configuration is an API, what's the harm in scripting against it?