Indeed. But to be fair, in today's world it is not only somewhat hard to imagine an OS that does not rely on things such as a filesystem with opaque binary files, directories, and hierarchical paths connecting these, but in order to properly communicate with the "outside world" you more or less have to have those things. Even AS/400 / IBM i added IFS, which is basically a bolted on hierarchical filesystem, to their system a long time ago by now.

Still sad that it has to be that way. I've long come off of thinking "everything is a file", and a file then just being a binary blob, is a good thing. (That's not even talking about other concepts from Unix we take for fully granted yet.)

Indeed. This reminds me of Rob Pike’s famous 2000 polemic “Systems Software Research is Irrelevant,” where he laments the decline in innovative operating system designs of the era.

Additionally, we need to consider the career incentive structures of researchers, whether they are in industry, academia, or some other institution. Writing an operating system is difficult and laborious, and coming up with innovative interfaces and subsystems is even more difficult. When a researcher’s boss in industry is demanding research results that are productizable, or when a researcher’s tenure committee at a university is demanding “impact” measured by publications, grants, and awards, it’s riskier betting a career on experimental systems design that could take a few person-years to implement and may not pan out (as research is inherently risky) versus pursuing less-ambitious lines of research.

It’s hard for a company to turn its back on compatibility with standards, and it’s hard for academic researchers to pursue “out-there” ideas in operating systems in a “publish-or-perish” world, especially when implementing those ideas is labor-intensive.

The widespread availability of Unix, from a source-available proprietary system with generous licensing costs to universities back in the 1970s, to the birth of FOSS clones and derivatives such as Linux and the BSDs, has made it much easier for CS researchers to not need to reinvent the OS wheel, instead focusing on narrower subsystems, but at the cost of discouraging research that could very well lead to the discovery of whole new ways of carrying things, metaphorically speaking. Sometimes reinventing the wheel is a good thing.

I still dream, though, of writing my own operating system in my spare time. Thankfully as a community college professor I have no publication pressures, and I get three months off per year to do whatever I want, so….

For sure. Also, we simply live in a world where computers, and their operating systems, are giant, impossibly complex structures. No single person can even fully know in all detail even what you would relatively consider a small part of neither any commercial computer, nor their operating system. In my job, I work with 15000 (!) page specifications, and that's only concerning one part of very many.

That pretty much guarantees that change can only be incremental.