Hello, I'm building a replacement for docker containers with a virtual machine with the ergonomics of containers + subsecond start times.
I worked in AWS previously in the container space + with firecracker. I realized the container is an unnecessary layer that slowed things down + firecracker was a technology designed for AWS org structure + usecase.
So I ended up building a hybrid taking the best of containers with the best of firecracker.
Let me know your thoughts, thanks!
Hey this is super cool. I've been researching tech like this for my AI sandboxing solution, ended up with Lima+Incus: https://github.com/JanPokorny/locki
My problem with microVMs was that they usually won't run docker / kubernetes, I work on apps that consist of whole kubernetes clusters and want the sandbox to contain all that.
Does your solution support running k3s for example?
we will evaluate. I created this issue to track this: https://github.com/smol-machines/smolvm/issues/150
Really appreciate the feedback!
Curious what prevented you from running docker in a microvm? I've successfully run docker in qemu and firecracker microvms.
With instavm (https://instavm.io), you can provide an OCI image built from a dockerfile.
What is the status of supporting live migration?
That's the one feature of similar systems that always gets left out. I understand why: it's not a priority for "cloud native" workloads. The world, however, has work loads that are not cloud native, because that comes at a high cost, and it always will. So if you'd like a real value-add differentiator for your micro-VM platform (beyond what I believe you already have,) there you go.
Otherwise this looks pretty compelling.
It helps if you offer a concrete use case, as in how large the heap is, what kinda of blackout period you can handle, and whether the app can handle all of it's open connections being destroyed, etc. The more an app can handle resetting some of it's own state, the easier LM is going to be to implement. If your workload jives with CRIU https://github.com/checkpoint-restore/criu you could do this already.
By what I assume is your definition, there are plenty of "non cloud native" workloads running on clouds that need live migration. Azure and GCP use LM behind the scenes to give the illusion of long uptime hosts. Guest VMs are moved around for host maintenance.
"Azure and GCP use LM behind the scenes"
As does OCI, and (relatively recently) AWS. That's a lot of votes.
Use case: some legacy database VM needs to move because the host needs maintenance, the database storage (as opposed to the database software) is on a iSCSI/NFS/NVMe-oF array somewhere, and clients are just smart enough to transparently handle a brief disconnect/reconnect (which is built-in to essentially every such database connection pool stack today.)
Use case: a web app platform (node/spring/django/rails/whatever) with a bunch of cached client state needs to move because the host needs maintenance. The developers haven't done all the legwork to make the state survive restart, and they'll likely never get time needed to do that. That's essentially the same use case as previous. It's also rampant.
Use case: a long running batch process (training, etc.) needs to move because reasons, and ops can't wait for it to stop, and they can't kill it because time==money. It's doesn't matter that it takes an hour to move because big heap, as long as the previous 100 hours isn't lost.
"as in how large the heap is"
That's an undecidable moving target, so let the user worry about it. Trust them to figure out what is feasible given the capabilities of their hardware and talent. They'll do fine if you provide the mechanism. I've been shuffling live VMs between hosts for 10+ years successfully, and Qemu/KVM has been capable of it for nearly 20, never mind VMware.
"CRIU"
Dormant, and still containers. Also, it's re-solving solved problems once you're running in a VM, but with more steps.
Somewhat related: I have a branch of Ganeti that has first-class ZFS support baked in, including using ZFS snapshot replication to do live migration without shared storage or CEPH: https://github.com/linsomniac/ganeti
Current status is I'm looking for more feedback. In a few weeks when Ubuntu 26.04 comes out I'm going to set up my dev/stg clusters at work with it, at the moment I've only tested it in a test cluster at home.
It works this way: It creates a snapshot of the zvol, and replicates it to the secondary machine. When that's done, it does another snapshot and does a "catch up" replication (the first replication could take hours on large volumes). Pause the VM, do a final snapshot+replication. Replicate the working RAM. Start up the VM on the new host.
Really appreciate the suggestion! By "live migration", do you mean keeping the existing files and migrate them elsewhere with the vm?
Thanks
I mean making any given VM stop on host A and appear on host B; e.g. standard Qemu/KVM:
This is feasible when network storage is available and useful when a host needs to be drained for maintenance.Live migrations and the tech powering it was the hardest thing I ever built. Its something that I think will come naturally to projects like smolVM as more of the hypervisors build it in, but its a deeply challenging task to do in userspace.
My team spent 4 months on our implementation of vm memory that let us do it and its still our biggest time suck. We also were able to make assumptions like RDMA that are not available.
All that to say — as someone not working on smolVMs — I am confident smolVMs and most other OSS sandbox implementations will get live migration via hypervisor upgrades in the next 12 months.
Until then there are enterprise-y providers like that have it and great OSS options that already solve this like cloud hypervisor.
I see. so right now smolvm can be stopped, and then "packed" (think of it as compressed), and restart on a different host. files in the disks are preserved, but memory snapshotting is still hard tbh
Ultimately the original does get stopped, but with additional techniques, we're talking milliseconds of downtime between when the old one stops and the new one resumes. (For live migration technology in general, no clue about smol machines.)
It's also feasible without network storage, --copy-storage-all will migrate all disks too.
What percentage of this code was written by LLM/AI?
For myself, I'd estimate ~50%
Not useful for things it hadn't been trained on before. But now I have the core functionality in place - it's been of great help.
[flagged]
Hey 'software engineer', how much of the output of an LLM it's actually reproducible vs the one from a calculator or any programming language with the same input in different sessions?
A lot vs a human? I bet the LLM with the same prompt will write same code as before more often than I would (given I don’t remember what I wrote in the past).
Why are you so concerned about the LLM producing the exact same code across different sessions? Seems like a really weird thing to focus on. Why aren't you focused on things like security, maintainability, UI/UX, performance?
Agreed. It's not like humans can produce the same output given the same input for anything more than trivial inputs.
I'd argue that it's actually a benefit; I like that I can do several generations and compare them and pick the best result. HP, for example, used to do this with software teams, and that's how we got Rocky Mountain BASIC (AIUI the competing team was East Coast BASIC).
Comp-sci people like repeatability when they want that and true randomness when that is desired. Things in between are rarely desired.
In computing, things are much more useful when they behave in predictable ways. Even AI, many (most?) would argue.
Computer science and software development are hardly related
Check the Plan9/9front papers or IWP's and say that again after reading the docs on GeFS for instance.
Thanks you proved my point
You are wrong. You may think that they aren’t related because you write software without thinking about computer science, but that says more about you and modern software development seat-filling than it does anything else.
One can’t write anything efficient without comp-sci being forefront in your mind the entire time you are writing. Which explains exactly why everything is so slow today.
If you’re not thinking about what is computable, what is not computable, and what is easily computable as you think about the problems you are trying to solve, you are a professional novice.
I studied computer science at one of the top colleges. It's true there's a lot of overlap between the two. But they are still very different fields.
You definitely don't need to study computer science just to avoid making common performance issues and in fact just studying computer science doesn't mean that you aren't going to introduce things like N+1 query issues, that's sort of thing comes from experience more than anything.
Likewise, just studying computer science does not mean that you are going to be well suited for software development in the real world.
> If you’re not thinking about what is computable, what is not computable, and what is easily computable as you think about the problems you are trying to solve, you are a professional novice.
Of course I consider all of these things but that didn't come from studying computer science in college lol.
Not really related to this 'discussion' but this is an interesting problem in the AI space. It's essentially a well understood problem in unreliable distributed systems - if you have a series of steps that might not respond with the same answer every time (because one might fail usually) then how do you get to a useful and reliable outcome? I've been experimenting with running a prompt multiple times and having an agent diff the output to find parts that some runs missed, or having it vote on which run resulted in the best response, with a modicum of success. If you're concerned about having another layer of AI in there then getting the agents to return some structured output that you can just run through a deterministic function is an alternative.
Non-determinism is a problem that you can mitigate to some extent with a bit of effort, and is important if your AI is running without a human-in-the-loop step. If you're there prompting it though then it doesn't actually matter. If you don't get a good result just try again.
Don’t know if this is an annoying response… but how about just going through the code and check and grade the quality yourself?
I could do, but the end goal is to scale this to 100x what I can do myself, and there isn't time to review all those changes. By attempting to answer the problem when it's tiny and I can still keep it in my head then I'll end up building something that works at scale.
Maybe. The point is that this is all new, and looking forwards I think it's worth figuring out this stuff early.
+1. i built something similar called shuru.run because i wanted an easy way to set up microVM sandboxes to run some of my AI apps, and firecracker wasn't available for macOS (and, as you said, it is just too heavy for normal user-level workloads).
Nice work on Shuru — I remember looking at it when I was researching this space. You went with a Rust wrapper on Apple’s Virtualization framework right?
I have been working on something similar but on top of firecracker, called it bhatti (https://github.com/sahil-shubham/bhatti).
I believe anyone with a spare linux box should be able to carve it into isolated programmable machines, without having to worry about provisioning them or their lifecycle.
The documentation’s still early but I have been using it for orchestrating parallel work (with deploy previews), offloading browser automation for my agents etc. An auction bought heztner server is serving me quite well :)
bhatti's cli looks very ergonomic! great job!
also, yes, shuru was (still) a wrapper over the Virtualization.framework, but it now supports Linux too (wrapper over KVM lol)
Is there a way to store configuration/data of applications running on a Bhatti VM on the host, ala Docker volumes?
Yes! Checkout the bhatti volume
They are ext4 blocks which exist independent of sandboxes.
Wow, that’s quite nice! I see that’s at VM time, any plans to add the ability to add them after the fact? I am really enjoying bhatti!
Yes, having a light-weight solution for local devices as well is one primary goal of the design. Another one is to make it easy for hosting, self or managed
What were the biggest challenges in terms of designing the VM to have subsecond start times? And what are the current bottlenecks for deceasing the start time even further?
No special programming tricks were used.
Linux was built in the 90s. Hardware improved more than a 1000x. Linux virtual machine startup times stayed relatively the same.
Turns out we kept adding junk to the linux kernel + bootup operations.
So all I did was cut and remove unnecessary parts until it still worked.
This ended up also getting boot up times to under 1s. The kernel changes are the 10 commits I made, you can verify here: https://github.com/smol-machines/libkrunfw
There's probably more fat to cut to be honest.
Any Windows support coming? My dev team is on Windows, this could be a game-changer.
Wsl2 runs a Linux vm so it definitely feasible and has the api's necessary.
It is on the roadmap, but frankly I haven't used Windows in a decade. I would love for a contributor to take that on as part of the free and open source spirit.
Sounds like it could be you? :)
>Sounds like it could be you? :)
I wish it could, but, like your housekeeper, I don't do Windows. ;-)
You could add OrbStack to the comp. table
Will do. Thanks for the suggestion!
The images or rather portable artifacts rehydration on any platform plus the packaging is neat. I have been working on https://instavm.io for some time around VM based sandboxes and related infra for agents and this is refreshing to see.
How is this different from lxd/lxc? How tied is this to kvm or could it work with other hypervisors like xcp-ng, VMware, or virtual box?
lxc/lxd are ~container managers, the kernel is still shared like docker containers. It runs on linux & needs to run in a linux virtual machine on macOS.
smolvm is a virtual machine with the ergonomics of containers like packaging and distributing, kernel is not shared. You'd run containers inside of smolvm.
Also, smolvm runs ontop of both kvm for linux and apple's hypervisor for macOS. So it's cross platform.
hi, great project! Windows support is sorely lacking, though. As someone working a lot with sandboxed LLMs right now, the options-space on windows for sandboxing is _extremely lacking_. Any plans to support it?
Hey, thanks so much! yah we will definitely add windows support later. We are exploring how to get this work with WSL and will release it asap. Stay tuned and thanks!
Not sure what level of sandboxing you need. Is Sandboxie not enough?
Yeah, it's in my mind.
WSL2 runs a linux virtual machine. Need to take some time and care to wire that up, but definitely feasible.
Just curious if this allow to sync file changes from host to vm which only Docker has that and no other VM alternative has that ability.
it has directory mounts - so access the same files: https://github.com/smol-machines/smolvm/blob/main/AGENTS.md?...
It also has file copy from host to inside the guest: https://github.com/smol-machines/smolvm/blob/70c97930dda5a92...
How is this different than Kara Containers?
kata containers is a container runtime that focuses on running containers inside a vm.
smolvm is a vm with some of the properties & ergonomics of containers - it's meant as a replacement for containers.