> There's no way to provide a good faith rebuttal here.

Indeed, there is no honest rebuttal.

> My entire argument is an extension of "LLMs can't be held accountable, so they must never make decisions".

BUT YOU CAN'T ENFORCE THAT. And whether they can be held accountable is IRRELEVANT. If a forest fire or an avalanche or a flood creates great destruction, it does no good to scream at it that it will be held accountable.

> If governments start letting them own LLCs without a human in the middle, we're in more trouble than "Who do you blame for this shitty code" or "Who's responsible for this compromise"

You're sooo close. You keep talking about blame and responsibility when IT'S NOT RELEVANT. It's trivial to create an agent that acts independently of any "human in the middle". People need to recognize that and act accordingly, and blathering about blame and accountability is profoundly confused.