Is it a stretch to imagine a compromised LLM writing compromising code? I've been operating under the assumption that LLM code might be bad and not working, but I hadn't thought about it compromising a system.
Yes and no. We used to have rampant script kiddies back in the '90s and early 2000s. After 2005-ish, well maybe 2010, most systems became adequate enough to not trust users' input, bug bounties, security as a separate role, etc, etc.
It would take at least some knowledge to hack, not just a random script from a forum.
Whilst true, security will improve rapidly again. Potentially via AI, potentially via other technologies.
If its easy enough to find exploits, its likely similarly easy to scan code for exploits, or use AI-based anti-virus technologies. The only thing holding us back is the cost of compute. We can't all run the latest models against everything.
The attack doesn’t use AI, but an AI could use the attack. (and the researchers definitely used AI to write the paper!)
Is it a stretch to imagine a compromised LLM writing compromising code? I've been operating under the assumption that LLM code might be bad and not working, but I hadn't thought about it compromising a system.
huh?
AI could enable script kiddies to pull this off.
Script kiddies have always been a problem you have to defend against. This is nothing new.
Yes and no. We used to have rampant script kiddies back in the '90s and early 2000s. After 2005-ish, well maybe 2010, most systems became adequate enough to not trust users' input, bug bounties, security as a separate role, etc, etc.
It would take at least some knowledge to hack, not just a random script from a forum.
Now, with LLMs, it's the '90s all over again.
Whilst true, security will improve rapidly again. Potentially via AI, potentially via other technologies.
If its easy enough to find exploits, its likely similarly easy to scan code for exploits, or use AI-based anti-virus technologies. The only thing holding us back is the cost of compute. We can't all run the latest models against everything.
Script kiddies will soon have capabilities that nation states once upon a time could only dream of.
That's largely true of most technology available today. You can buy a cheap $100 phone that's more powerful than old super computers.