> LLMs don't get a free pass because they're also so good at writing typescript for beige CRUD apps and bedtime stories.
Plenty of useful things get free passes to be dangerous. Traffic accidents are the leading cause of accidental deaths in 11 states, but we don't ban cars because they're dangerous. There's plenty of safety features, but we acknowledge and accept that people will die. People like to pretend that they won't sacrifice safety for convenience, but they continue to do it time and time again.
Yeah, and in those cases there is strong governmental regulation surrounding what a "safe car" is. We don't have that with AI. Another analogy is with weapons manufacturing; American weapons manufacturers have some, but relatively little, regulation when it comes to selling weapons to Americans; but they are subject to significant regulation when it comes to shipping those weapons overseas.
We need legislation that empowers a Federal agency adjacent to the CDC or FTC with the power to enumerate specific capabilities models could exhibit that we deem dangerous, and require model manufacturers to guarantee that their models cannot exhibit these capabilities. The reality is, zero of the safeguard systems invented by the frontier labs today are sophisticated enough to do this. The labs are extremely, extremely bad at Safety, relative to both how much impact their products are having on the world and how good other industries like Medicine and Manufacturing have gotten at safety. I'm talking a total and complete culture shift.
Google had a "appoint Eric Schmidt as CEO" moment like this. There were kids running it before; make them rich, and retire them to an island somewhere, because the impact your products are having on the world cannot possibly be allowed to continue with your current leadership at the helm. Dario is that problem now. I think Sam/OpenAI can adapt and mature. I have zero faith that Dario and the furry EA cult in Anthropic leadership will. So, this is what being unable to mature looks like; the public will make your company safe, one way or another.