I think the attacks generated by Meta, Open AI and Anthropic prove that large corporations are not responsible enough to be trusted with advanced AI, so we should ban all commercial AI services and only allow open source models that are in the hands of hobbyists and individuals -- hobbyists and individuals that have so far proven to be much more trust worthy.
Not saying you’re wrong, but I think the bigger issue is how easy it seems to be for models to hack companies, even ones with generally ok security. Most tech companies are not doing continuous, deep security audits of their code and infrastructure. Dependencies are not updated quickly as RCEs are discovered. (And any org with a slow release process where it’s hard to be confident that an OS or package update won’t break something… is in even more trouble.)
The only reason more companies aren’t exploited is because human attackers don’t have the time and energy to waste on trying every play in the book, or attacking lower value targets.
They key lesson I've picked up from the past ~4 months is that models are now good enough that, if there's a security hole, they'll brute force their way into finding it.
The only solution that makes sense to me is for defenders to get to point these models at their own code to find the holes before the attackers do.
But that's hard, because how do you limit access to defenders and restrict access to attackers? Attackers aren't exactly honest people.
These attacks are also incredibly loud. Many attackers are motivated to operate very quietly. We haven't seen any tradecraft from these machines, it's all noisy and bombastic.
When we see them mount a quiet backdooring campaign, like the XZ-SSH attack, or something like Stuxnet, then we'll have real problems.