I’ve tried so hard to figure out what the hell oxide even is, but the website is utterly impenetrable. I guess it’s an “integrated platform” and does “AI” and it’s “purpose built for frontier workloads”, but… I have absolutely no idea what it is, still.
I guess it’s a bunch of computers. It says “AMD” in a bunch of places so, I guess they’re x86. Where are the GPU’s? If it’s for “frontier workloads”, wouldn’t those be sorta important? Is it its own OS? No idea. I see people adjacent to Oxide mention IllumOS sometimes[0], so does that mean it’s a solaris-like OS? Why would I want to run k8s (presumably with linux containers) on a not-linux OS? Or does it virtualize linux instances?
Do actual CTO’s go for this type of marketing, with zero details and nothing but pure fluff about “solutions”?
[0] This of course is from HN comments, I see no mention of any operating system whatsoever on the website, so if it is IllumOS, it’s not like you can easily find that info anywhere.
It's a rack-sale computer that uses hardware and software co-design to achieve things a stack of disparate 1U servers and network switches can't. You plug in power. You connect it to your network. You get a cloud provider like control plane API and web console to provision VMs, disks, and VPCs. That's it papa.
The host OS is an implementation detail since you, the customer, aren't running workloads directly on the host OS. The VMs running on Oxide run on our host OS, much like other cloud providers.
That rack-scale hardware and software co-design allows Oxide to provide higher CPU and memory density in the same footprint at lower power than competitors. That's super helpful for companies operating at scale where data center power matters. The co-design allows us tackle security problems by eliminating the BIOS, providing attestation from the firmware up to the guest VM, and actually updating the firmware that's on Oxide rather than letting it rot like most customers do today.
This is why when people ask which verticals Oxide targets we kinda say "all of them" because different customers benefit from different Oxide features, but all customers need the core VM, disk, VPC abstraction. Some customers come to us because they don't have an API to manage their on-premises compute and Oxide solves that. Others come to us because they need absolute confidence that there's no malicious firmware running in their compute stack. Others come to us because they are tired of paying exorbitant amounts of recurring money just to run on-premises compute.
Our job is to make Oxide an appealing on-premises computing platform for customers to run their public cloud provider workloads and on-premises workloads without it feeling like it's an entirely different platform than you're used to.
Here's the docs:
https://docs.oxide.computer/guides/introduction
I wrote this back in 2022, but it's still relevant https://news.ycombinator.com/item?id=30678324
It's a private cloud. I guess you might not be a target customer.
I develop cloud infrastructure controller software for a living but sure. I’m certainly not a target customer, but as someone casually interested in this space it’s certainly very light on details.
Re-assessing my criticism, I think I’m mostly complaining that I can’t figure out what software it’s running. But I guess the target customers don’t care, since it’s an implementation detail. I’ve only ever heard “oxide” in the past in reference to the OS they were (are?) developing, which a web search says is called “hubris”, so when I see an article like OP I’m wondering “are they running k8s on top of hubris? Wow!” But I’m imagining that’s very much not the case.
In the comments in this very section an oxide employee mentions they’re using Illumos, but I don’t see that anywhere on their website either.
But you’re probably right in that the target customer doesn’t give a shit what OS it runs, so long as they can get instances deployed (although I would question why, if you’re going to run k8s anyway, you don’t just run it on bare metal and skip the hypervisor, but that’s my bias showing up as someone who writes bare metal controller software.)
You're completely skipping the management of the bare-metal hardware itself. Sure, you can buy a stack of whatever commodity hardware you want and run Kubernetes on bare-metal. What about updating the firmware on that hardware? What about expanding and the hardware you purchases is longer available? Now you have more disparate hardware that may not even be compatible with the software you're running. Those problems exist at scale and is what Oxide is tackling. It's more than just buying a computer and set it and forget it.
I'm not really criticizing oxide as a product here, I'm sure it's great. I'm criticizing the website for being heavy on "solutions" and light on details. But really, the main detail it's lacking is the operating system it uses, so as I said, I needed to refocus the criticism.
But to your point, I don't think I ever said machines don't need a management control plane (I develop a bare metal management control plane for $DAYJOB, I'm fully aware of what it entails!) I'm saying that you don't need to insert a VM layer between the bare metal and containers: Having k8s run on the bare metal OS would be ideal IMO.
But since Oxide seems to be Illumos-based, that's basically not possible... At least not for containers as most people know them (ie. with linux-based docker images.) Heck, even if running a VM layer between the OS and the containers, it looks like running your own hypervisor prevents you from (currently) having nested virtualization, which makes certain k8s workloads a lot harder.
"Light on details" is not often a criticism that we get, but I think you might be looking in the wrong place: for the kind of detail that you're looking for, you should be heading into our RFDs -- specifically (for the issue you seem to be most interested in), RFD 26.[0][1]
[0] https://rfd.shared.oxide.computer/rfd/0026
[1] https://news.ycombinator.com/item?id=41515447
Its funny because Bryan has spend a good part of the last decades walking around and telling people that they should have VM layer. And this criticism can credibly made against his company.
Interestingly he has mentioned that in the podcast and basically said, the industry has standardized on VM and in the kind of environments they run, there just isn't a good way around it.
But yeah, a world where Illumos based bare-metal containers had become the standard would be a nicer. But its not where the market is.
I said some of this elsewhere, but the operating system is an implementation detail since we're not selling you bare-metal compute--we're selling you on-premises cloud computing. I think you were looking at this from hardware plus operating system rather than hardware plus a cloud computing control plane.
I get what you're saying though, and you're right that certain workloads like Kubernetes on the metal that can take advantage of hardware wouldn't be a good fit for Oxide today. We could decide to change that in the future, but I don't see that as a priority for Oxide given all else we have to build first.
(To answer one of your specific questions, "hubris" is an embedded RTOS and is what runs the BMC-like service processor, and illumos is the host operating system.)