Ah come on, guys, let's talk pragmatically. "Malleable editor as an OS layer" has benefits beyond subjective reasoning. Emacs has had M-x shell-command and arbitrary elisp eval forever. A metacircular MCP isn't some new capability class. Even if I didn't use Emacs - my shell, my editor, my browser extensions, my npm install, my VSCode plugins, my curl | bash from yesterday - they all have the same access. Singling out the LLM in this context is like selection bias.
Of course, reasonable mitigations are a must - just like for any other tool. Narrowing MCP scope - tool routing rules, read-only git defaults, etc. "Docker or nothing" is a lazy answer - Docker-for-everything has real costs: friction, broken integrations, worse ergonomics.
Practical security is all about staying in the goldilocks zone. You shouldn't get relaxed about the basics - sandboxing, 2FA, password managers - they are worth doing, and you can get so paranoid about so many things, and yet against a targeted, well-resourced attacker, your sandboxing posture is mostly irrelevant. The interesting attacks bypass the threat model entirely. Read about Ben Nassi's team research¹ - pretty cool example. There are multitudes of other ways and your Docker container won't stop them. Defend against the boring 99%, and accept that the 1% is someone else's problem (or a much bigger problem than your dev environment)
—
¹ https://www.nassiben.com/video-based-crypta
TLDR LLM Summary: Researchers showed that a device's power LED subtly flickers in brightness and color while the CPU performs cryptographic work, and these flickers leak information about the secret key. By pointing an ordinary video camera (an iPhone or an internet-connected security camera) at the LED and exploiting the camera's rolling shutter, they boosted the effective sampling rate from 60 to 60,000 measurements per second, enough to do cryptanalysis. Using only this video footage, they recovered full ECDSA and SIKE keys from a smartcard reader and a Samsung Galaxy S8, with no malware on the target devices.
There are many better sandboxing options than docker (in terms of security and/or ease of use), and it sounded like you weren't doing sandboxing.
It's your computer and you can do whatever yolo nonsense you want, my dude, but put those goalposts back where they were.
"Don't run that shit on a credentialed box with data you care about" is addressing real threats, not some goofy nation state thing or abstract security research.
If you let the footgun machine constantly generate new code and run it on your computer, you're just asking for data loss and bad shit to happen.
Docker isn't a great solution but it at least doesn't let yolo code delete files or access env vars or read the contents of .ssh/
> my browser extensions, my npm install, my VSCode plugins, my curl | bash
Yeah, and you shouldn't yolo those, either lol. If they didn't come from a trusted source, you need to read through them. If you don't want to, don't use them. That's not paranoia, that's, like, normal.
> If you let the footgun machine constantly generate new code
Are you talking about autonomous LLM projects that automatically write code? Yeah, no shit, I wouldn't run anything like that directly on any machine without sandboxing. My typical LLM use inside my editor is never in self-driving mode, there's not even cruise-control - I tell it exactly when to write, where to write and how to do it. Automated scripts never get run by LLM and don't get to run at all without prior precise and meticulous inspection. I'm not moving goalposts - at worst we're in disagreement on the level of pragmatics vs. paranoia, that's all.
I don't even get why people are so crazy about LLMs generating code - on both sides. LLMs for me personally are such a great tool for investigating things, for finding things, for bridging the gaps - the stuff that happens 10K feet above code writing. By the time I'm done gathering the details, code generation becomes an almost insignificant touch of the whole endeavor.