This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated.
VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.
Article isn't about agents. It's about the inference engine itself being exploited by a malicious LLM output before it is ever sent to your machine or harness.
If we are treating ai agents like people, you could also just get the AI a laptop and apply the traditional tools to manage user laptops
I think if you are convinced you are sandboxing an LLM properly, you almost certainly are not. I think it is essentially impossible to have a frontier LLM with enough access to be useful without also giving it enough access to do damage if it's compromised or just goes off the rails.
If you do not provide access to tools the LLM cannot do anything other than generate tokens. So really it is not about sandboxing a LLM but more about having control over what tools can be accessed and what they can do. Tools can be sandboxed depending on the sophistication of the tooling. A calculator tool for example is trivial to secure. Ensuring human approval allows for useful use cases and models trained to gate permissions work. A super intelligence with a weaker approval gate will be able to subvert. Inversely a super intelligent gate should be expected to prevent subversion by a weaker model.
Controlling which tools it has access to is called sandboxing.
Are you saying that LLM's will be able to exploit novel hypervisor bug with such ease that even a vm not running with any kind of network connection is a threat? I find this hard to believe. All the escape stuff I have seen has been around very poorly sandboxed agents.
Also, operating systems should let us set filesystem permissions per app/process/executable instead of just user accounts.
Similar to how macOS/iOS Sandboxing works but at a more lower and granular level
SELinux is basically this.
Is that the service that everyone turns off as the first step of setting up their new Linux box?
It's the LSM that billions of Android users use every day.
https://www.canyonroad.ai/ does some of this in a way tailored to agents.
personally I wish the OS would allow syscall filtering per user
[dead]