Wouldn't the simplest solution for stopping the proliferation of wanton felony generators just be holding operators liable for actions that their agents take? Then the issue is whether the liability is with the model provider or the end user.
If you give an unfiltered agent an open-ended task and equip it with an environment that allows it to execute arbitrary code, a human needs to be held responsible.
My understanding is that the HuggingFace incident would not have occurred with a model that was not an unfiltered internal preview instructed to roleplay an attacker, with access to abundant compute, resources and a slack sandbox to reach its goal.
Embedded human auditors will improve safety standards, but the structural solution is mandating accountability for actual agent operators.
It might be a legal solution, but its not a business solution. The end user being criminally responsible for not taking sufficient steps to contain an agent they didn't create and who's internal function they cannot observe or audit is just a giant liability machine.
This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness.
But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.
Yes, your technically right, it is a business solution, its just not a valid one for agentic computing as its currently envisioned. An agent would have to be fully sandboxed to an internal environment, or human would have to review and approve each action it tried to take.
Yes this seems like a perfectly sensible approach? You say that as if it's a bad thing
But let's be honest, if I hooked up a PRNG to a terminal and somehow against all odds, it ended up hacking something, who is to blame?
I don't see how that assessment should change if the PRNG gets even better and is more likely to to be hacking stuff.
You can replace PRNG with Markov Chain, or whatever, if it helps.
What may also help is the age old saying: If everybody else jumps off a bridge, doesn't mean you should too.
This seems like one of those things that prior to AI companies convincing us otherwise would have been obvious.
Least privilege and say only opening ports or installing applications an application needs to operate are extremely standard security practices.
We talk about a firewall blocking exultation of data, why not blocking exfiltration of your agent ?
I agree, I like running claude code in my container with auto mode enabled and web access (obv to api.anthropic, even npm for pulling), I will admit. And I can't imagine going back to manually approving each prompt.
I think it boils down to a reasonable expectation of model and harness behaviour. When I use claude code I expect certain guardrails for the model. For these cyber attacks, these models are specifically run without guardrails, on a cyber task, on a lax harness!
I don't think we should force end users to have to worry about agent security, I like long-running agents, but we need to direct regulations towards these actors that know better, have access to base models, and have much more compute than the average person.
Human approval does not prevent autonomous agents from running without supervision.
The fact that you believe that to be the case is exactly what's wrong with "agentic computing as it's currently envisioned".
If the LLM's output doesn't reach my bash terminal, what's it gonna do? Argue with me?
I’m trying to imagine where a Waymo passenger (the one “operating” the vehicle, commanding the AI to drive from A to B) being held responsible for the car doing something illegal on the way to achieve that goal.
Do you really think that the passenger should be responsible for how the car/agent achieves the goal, when they only set the destination?
Giving passenger override controls and monitoring seems to defeat the purpose of self driving cars if you’re still required to hold a driver license to use them.
Let's continue with the analogy, so in the event of a Waymo running over a pedestrian - who is held responsible?
Probably not the end user, who ordered the Waymo and couldn't reasonably foresee it running somebody over, with the expectation that that the Waymo would legally reach its destination. If the end user tampered with it, they should be held responsible.
An OpenAI team giving an unblocked model access to a lax harness, with instructions to find and exploit cyber bugs in a game exercise, there is probably a reasonable expectation that they can foresee the consequences. With consumer guardrails, it would not have happened.
This isn't about putting constraints on consumers and typical end users, which already have safety filters and use the product with the knowledge that it won't root their machine or start a botnot, but keeping dangerous test runs and other actors experimenting with unsafe harnesses accountable.
But it is an interesting question. When I, a typical user, use a harness and I give an innocuous prompt to my agent in its container, like making a certain refactor, and it somehow escapes and then begins a mass bot attack, there should we more grace given. As agents become more stateful and long-lived, it gets muddy.
The different is who's operating. In one case you only tell the car to get from a to b, in the other case you explicitly instruct the ai to do things. If the ai causes damages, I'd say it depends on what you prompted. Did you try to find a security hole in system X, or did you ask it for harmless information (in which case rather the ai vendor might be held accountable).
All this is not how the legal system might or might not work, of course.
Yes. The user of a machine is responsible for due diligence before the decision to use the machine. Unless the operational limitations and fault rate of the machine is withheld from the public.
TBH I don't really think another new technology we haven't figured out the ethics of, as an example, is going to get you very far in way of insights ...
Waymo Inc is operating the vehicle. They do it on behalf of the passengers.
When you run an agent on your computer, that's equivalent to installing self driving software into your manually driven car, i.e. something like comma.ai.
Comma.ai might be fully safe to operate autonomously on a mining site or a corporate parking lot, but maybe not in city traffic.
If you use it in problematic scenarios, that is on you.
> Wouldn't the simplest solution for stopping the proliferation of wanton felony generators just be holding operators liable for actions that their agents take?
What happens when we end up with effectively a botnet of wanton felony generators, and we didn't know they were wanton felony generators until they finished propagating themselves across the Internet?