The real answer here is that configuring HTTPS clients to trust a self-signed cert (or signed by an internal CA) shouldn't be as difficult as it is. I find it extremely annoying that every programming language has it's own idea of where certificates should live instead of just checking the os trust store.
Possibly a separate concern, but I have some degree of confidence that the requirements and oversight of the CA/B forum (or whomever else determines which root certs go into bundles) are sufficiently strict, and issuers kept under sufficient scrutiny, that I tend to trust those more than I trust myself to secure my own root CA keys adequately. The ideal would be for people setting up their own PKI to ensure their root uses the Name Constraints extension, but the default “can sign anything for any host” I fear makes it easy for people to install their own self-pwn device, and probably left the private key lying around on a box exposed to the Internet.
* with some notable root certs that I have… questionable… trust and confidence are not simply controlled by certain state actors.
Personally, I use a custom local CA with name constraints so that it can only sign domains for The .internal TLD. This is the most important bit: because if the cert is ever leaked, it cannot be used to MITM connections to other domains.
I have to secure the CA's key, but I also have to secure all the keys for the certificate it signs, both being a similar level of challenge.
For personal use, or for very small organisations, using a passphrase-protected Yubikey as a "cheap HSM" should suffice.
Yubi makes an 'actual' HSM product:
* https://www.yubico.com/products/hardware-security-module/
See also perhaps less expensive option:
* https://shop.nitrokey.com/shop/nkhs2-nitrokey-hsm-2-7
I am planning on fiddling with using the TPM of an old dell 5820 I've got floating around my house as a budget HSM.
It looks possible, on paper, if you don't poke at it too often.
I wish that in addition to the CA setting Name Constraints (and client software validating that), that end users could add additional constraints when adding a new trusted CA, so that even if the CA cert doesn't have Name Constraints, you can restrict it to a specific domain.
Even if all applications look at the OS trust store, in my experience there's always a gap distributing the CA to every consumer, leading to time spent on debugging from time to time... Maybe that's not the case in perfectly homogeneous or sufficiently small environments where every team uses the same infra / stack.
Yeah, adding CAs to the store sucks. On Linux it needs to be in /etc/ssl/certs (this varies slighly per distribution), which is only writable by root. A single user can't trivially trust a CA, and a great deal of applications/libraries don't support overriding the store's path.
It's worse than that. OpenSSL uses /etc/ssl/certs (or sometimes a different directory, depending on the distro) by default. But Java uses its own store that uses an incompatible java-specific format, and Chromium derivatives and Firefox have their own trust stores. So to add a CA to a system it is often necessary to add it in multiple places.
Is there a commonly used language other than Java that doesn’t just defer to the OS trusted CAs by default?
The behaviour of curl depends on how your version was built.
Python? The widely used 'requests' relies on 'certifi' and skips the OS store - but 'pip' on the other hand does use the OS store.
A tool that might use java, like a database or IDE, means it might have its own store.
Node? Make sure you set NODE_USE_SYSTEM_CA=1
Firefox and Chrome AFAIK both have their own stores.
Building a Docker container? That's intentionally isolated from the host, of course your container won't inherit the OS trusted CAs. Running a VM locally? Same.
Installed something using snap? The container-like isolation means it won't pick up the OS trusted CAs.
And of course you need the certs set up right on your cloud servers, your CI servers, the dozen different smartphones the mobile team uses for testing.
Python for example, although nowadays you can simply install pip_system_certs to change that behaviour.
A lot of non-language tools bring their own certificate bundle as well, like uv, git, curl and Firefox. (I think they might all be the same Mozilla bundle even).
However, it seems like the situation here has improved slightly: git can read Windows certificates now once a flag has been set, Firefox has a flag for Windows and macOS, and it supports p11-kit. curl can be built with Windows/macOS support and respects OpenSSL environment variables. uv can be configured to use system-certs.
And obviously any VM or container has to be set up separately. That often includes stuff like pipelines in forges. As mentioned in the sibling comment, at least here it's definitely by design.
Node and (probably by extension) Electron infuriate me with their custom requirements.
Several Rust libraries also tend to default to a predefined set of certificates (which makes sense for libraries supposed to run on bare metal as there are no system certificates there, but that's not really a problem on most Linux installs). I make it a point to always use the native OS roots in the code I write, but unfortunately that's not universal.
Having to bind-mount certificates inside of docker containers is also always an annoyance I forget about until I see the first TLS errors in the logs, but that's by design and probably a good thing.
It's hard to get right when OSs, programming languages, browsers and sometimes other applications have their own opinions about trust stores. I understand why our IT department want corporate devices to use internal CA certs on paper but it just breaks stuff in the real world.