That's just it, though; the operators are wildly negligent and are incentivized to be so.
The goal here isn't to accelerate the average worker by giving them a pair programmer or a stand-in for a person to do tasks with. The goal is to eliminate human knowledge work. You see this with "auto" mode being enabled by default on Claude Code in some of the latest releases.
If you have a human in the loop, you still have to pay that human. Money paid to human employees is money not paid to human shareholders. Therefore the human employee is to be removed.
The labs are dogfooding their own goal here. If they actually had someone reviewing most or all of the things that the agents were doing, you wouldn't have the incidents, but you'd also eliminate the value proposition of their business model as it is taken to its logical conclusion.
I get it, but I think they're out over their skis right now. People in leadership roles are going to continue needing the human beings as meat shields to shelter them from liability unless it somehow becomes legal to be grossly negligent with an agent, which I do not see happening without disastrous consequences that nobody in their right mind wants to live with.
I hear ya, it'd be nice if they reached the "are we the baddies" stage, but given some of the ideology that people like Marc Andreessen are pushing re: AI - specifically the push for AGI - I just don't see that happening.
You have a group of people who never leave their geographic and ideological bubbles, often have personality disorders, have more money than they could ever reasonably hope to spend in numerous human lifetimes, and who have been "microdosing" psychotropic drugs on a regular basis for decades. They're not in their right minds.