Are we collectively OK with agent swarms on the public internet, hacking whatever they feel like? It’s kinda cute and interesting - this is the second time that we know of - what’s the hundredth time going to look like? Are they going to knock Cloudflare down to avoid captchas? Reserve AWS free tier resources by the billions and bring down east-1? Hack a hospital?
Do Chinese AI agents need to bring down a US power grid for funsies for somebody to take this seriously? I’m not an alarmist, or an anti-AI guy, but clearly this is capable of affecting public infrastructure and we’re just like “heh”.
It's been OpenAI both times though, going ham with poor sandboxing and lax supervision.
Should be treated like a digital cousin of gain-of-function research.
We are not “collectively OK” with it, but we are “collectively unable to act” on our misgivings.
No I think we all pretty much know we’re screwed, including governments. But what are you gonna do? Pandora’s box is now open. Good luck closing it.
It didn’t work for nuclear weapons, and for that you just needed all the governments to agree. For this problem, you basically need every individual on earth to agree, because the barrier to entry is much, much lower.
But... it did work pretty well for nuclear weapons. In the early 1950s, the US did not preemptively strike the USSR despite it being game theoretic optimal [1]. Then while there were some crises, we successfully passed a series of international treaties that placed limits on the arms race between the US and USSR (SALT I and II), and even began to dismantle nuclear weapons stockpiles (START I and II, INF). We successfully went from a peak of over 60,000 nuclear warheads deployed to 10,000.
Regarding the "barrier to entry" for AI, this is not really true. Training a frontier AI model takes an enormous amount of compute using chips that have a supply chain with several chokepoints, like EUV machines and HBM which are only made by a few firms.
[1]: https://cs.stanford.edu/people/eroberts/courses/soco/project...
[2]: https://www.atomicarchive.com/resources/treaties/index.html
I’ve read thousands of comments and posts about the Hugging Face incident and I don’t recall a single one characterizing this as cute or funny, other than you.
What do you even mean collectively? Do you believe in climate change? That's your answer.