Yep, this is my experience too. I think of it more as a very, very smart and fast intern -- you can tell it’s going places, and in many ways is already way better than you, but it still needs an experienced hand to steer it.

My rule of thumb is, any special processes you put in place for AIs are either sensible for humans as well, or they’re not worthwhile. Good CLIs, auto-summarization of long command outputs, Markdown docs and workflows -- those are all useful for people too!

To guard against mistakes and abuse, you use sandboxing and scoped permissions, not micromanagement.

One thing I’d like to figure out is a good pair-programming workflow for AI agents. You can tell a high-level model to go and do something, and that works; you can use a low-level model as an IDE assistant, and that works; but they’re separate workflows. What would be really useful is a way to kind of hand the keyboard back and forth with the high-end model and build something together. But safely, not in full-on YOLO mode on my own machine. This is one specific area where humans and LLMs differ -- it’s so much faster than me that I can’t just grab the keyboard back from it if it goes off the rails.