> That the answer is to just leave AI labs to carry on as they are?

Yes, and no. I do not think there's a need for special treatment of LLMs. But the labs certainly shouldn't be allowed to go around hacking things on the internet when they could have prevented it by simply taking time to ensure sandboxes they run test models in are given more than a quick look by a junior engineer to configure.

But the interesting thing is that there are laws that already govern this. They're just, for some reason, are not applied to Anthropic and OpenAI. Cyber crime is already punishable, and both labs did, in fact, commit cyber crimes. Did the existing regulations fail? No, the enforcement did.

The same goes for bioweapons. Any chemistry undergrad that can read an openly available paper can manufacture insane things with just common household ingredients, they do not need LLMs for that. This is a made up scenario with no evidence whatsoever to support it. There are millions of people that could poison your water supply today, without ever touching an LLM.

If we're afraid of LLMs turning into superintelligence, and we think superintelligence shouldn't be available to anyone at all, then we must all agree not on pacing the frontier, but just stopping any and all research into this. And that's not going to happen.