The author dismisses many entries, saying "it's a programming language, I want a config language, or I'd use Javascript". I completely disagree with the author, given my experience.
- You want bits of programmability in your config language. Else you end up with copy-pasting pieces and adding comments like "must be the same value as foo.bar below!". This sucks enough that you may end up with a config generator for some custom meta-config language, which you need to produce and maintain. You very certainly need symbolic constants, it's great to have simple arithmetics and string concatenation, and ideally you need a way to map a config fragment over a list.
- You don't want a Turing-complete language as your config language. You also don't want to allow any I/O or other effects in that language. You don't want to allow your config parser to do anything but to form a tree of keys and values in memory after reading the config. Javascript is right out. XML is also right out, because entities allow for external resource links, that is, uncontrolled I/O; this was a source of many exploits.
- Nevertheless, you want your configs to be modular when needed, allowing to graft into your main config imported values from some common templates, specific overrides, etc. Direct inclusion is a poor but serviceable approach.
- You want the values in your config be typed. They can be strings by default to save some typing. But you want your config parser to be able to complain when you write a string instead of a number, of a float instead of an integer, or an integer out of representable range (the bane of large numbers in JSON), or a date that cannot be parsed, or a null for a value that cannot be null. This saves you from a ton of nasty surprises when configs are updated.
- You want your config language to have explicit delimiters, so that incompletely copy-pasted constructs would be detected. Hence YAML is a bad choice, JSON5 is a much better choice.
- Of course, you absolutely want comments. JSON is a serialization format, not a human-operated config format.
This is why, I think, CUE, Dhall, and Jsonnet are doing more or less the right thing, but they are a tad large. Maybe something simpler and more compact exists, with some limited programmability and easy syntax.
The features you mention might be useful, but they're certainly not required.
If your program has grown so complex that users would benefit from modular configs, references, and so on, then a configuration file is the wrong approach for configuring it. You should rather provide a custom UI for users to modify the program's behavior, and persist the configuration transparently in the background for them. This can be done via a CLI, GUI, API, whatever.
The fact that tools like GitHub Actions, Ansible, Kubernetes, etc., abuse a format like YAML to make it work as a DSL, and we needed to invent tools like CUE, Dhall, and Jsonnet to manage this mess is absolute insanity. Most programs don't need such level of complexity. And for those, plain JSON, or minimal configuration formats like the one discussed here, are good enough.
> This can be done via a CLI, GUI, API, whatever. So then I need to write a Terraform provider or whatever and a bunch of my own configuration to be able to declaratively configure the thing?
For the reasons you mention (mainly the first one), it can sometimes be useful to use a separate programming language with functions that are used to create the required format, without needing the configuration format itself to do these things.
Can you do an infinite loop in that separate language? Throw an exception? Write to disk? Start processes? Access the Internet?
If not, you can use this language safely inside the config.
If yes, you could as well use the main language of your application, and not introduce an extra dependency.
There is, of course, always a temptation to take awk, or PowerShell, or Ruby, "because it's already installed on the system", which may not be the case on all platforms.
I did not mean that you have to introduce an extra dependency; the programming language used to make such a configuration can be the main language of the application. Although whoever needs to configure it can be someone other than whoever wrote it, and they can use whatever they want to use. The output can then be used as the configuration even on a different computer if it is not necessary to change the configuration but only use it on multiple computers. This will be appropriate for some uses but for other uses it will not be appropriate. Note also that different programs will have different requirements for what kind of configurations you will require.