the agentic shift is where the legal and insurance worlds are really going to struggle. we know how to model human error, but modeling an autonomous loop that makes a chain of small decisions leading to a systemic failure is a whole different beast. the audit trail requirements for these factories are going to be a regulatory nightmare.
I think the insurance industry is will take a simpler route: humans will be held 100% responsible. Any decisions made by the ai will be the responsibility of the human instructing that ai. Always.
I think this will act as a brake on the agentic shift as a whole.
that's the current legal default, but it starts breaking down when you look at product liability vs professional liability.
if a company sells an autonomous agent that is marketed as doing a task without human oversight, the courts will eventually move that burden back to the manufacturer. we saw the same dance with autonomous driving disclaimers the "human must stay in control" line works as a legal shield for a while, but eventually the market demands a shift in who holds the risk.
if we stick to 100% human responsibility for black-box errors that a human couldn't have even predicted, that "brake" won't just slow down the agentic shift, it'll effectively kill the enterprise market for it. no C-suite is going to authorize a fleet of agents if they're holding 100% of the bag for emergent failures they can't audit.
Yes, that is why I strongly support sticking to 100% human responsibility for “black-box” errors.