Next up:
Obvious pattern of using ai to replace human reasoning in a proven methodology of malware distribution, C&C, and network infiltration obviously possible, say researchers.
Researchers use AI to create the torment nexus using commodity hardware, demonstrating the very real threat that AI could enable attackers to create torment nexus nodes using commodity hardware. “It wasn’t even that hard !“ says one researcher. Firmware available to qualified researchers who pinky swear that it will not be leaked.
Researchers set fire to laboratory with gasoline, killing seven volunteer victims, demonstrating that laboratory fires are a real risk and can carry significant consequences, especially when gasoline is involved.
Just because you can, doesn’t mean you should.
this is part of the pro-active security loop. gotta demonstrate how it can break to figure out how to defend it.
our other choice is to let someone else figure it out in relative secrecy. then theyre able to cause a bunch of damage to a wide range of systems. with no defences for it. everyone would be scrambling around figuring out how to deal with it while the damage is going on. not good.
I’m totally onboard with (and an adamant user of) proactive security. But there are classes of threats that are obviously possible, and the -concept- does not need validation.
Now , a control anchored experiment with balanced and unbalanced attacker/defender LLMs, that would be instructive and useful.
The idea that an LLM can deploy other LLMs on a machine it has access to is not research. Neither is the idea that an LLM can autonomously infiltrate and expand its access over a network. I have already done both, and it’s literally just a couple of prompts and a pile of reference docs. I use LLMs to deploy LLMs on my infrastructure, and I use LLMs to analyze security vulnerabilities on my networks, including deployment of access ladders on vulnerable machines. That is SOP, not research.
If they had used a pair of identical experiments, one that was exposed to an infiltrator LLM, and the other occupied by a defensive LLM and then exposed to the same threat, that would be an actual experiment.
As it is they just threw a roadflare on a dry field, and yup, Dry fields burn. They at least could have done it with and without recent rain.
They published only the obvious and dangerous part, none of the hypothetical or potentially useful part. Low effort, rush to publish.