This really isn’t that hard to imagine with even an average level of creativity. Any critical system that people’s lives depend on can be in the crosshairs, whether by accident or not.
This really isn’t that hard to imagine with even an average level of creativity. Any critical system that people’s lives depend on can be in the crosshairs, whether by accident or not.
People say this, and then the scenarios they come up with are boring and uninteresting. 80% of the scenarios in this thread are "AI hacks something", as if hacking didn't exist before agents, and the remaining 20% are mostly made up of other things humans are already trying to do at scale.
Why am I supposed to believe that a virus created in a virology lab with AI assistance is uniquely more dangerous than a virus created in a virology lab without AI assistance?
> Why am I supposed to believe that a virus created in a virology lab with AI assistance is uniquely more dangerous than a virus created in a virology lab without AI assistance?
You're not.*
The difference is how likely this is to be done, not how dangerous it might be if it was done.
And this isn't likelihood in a 0-100% sense, but in a Poisson distribution sense, i.e. mean time between incidents.
Remember: these AI minds aren't that good with the physical world, and have nevertheless been repeatedly connected to robots, and now some of the AI labs are showing off their actual wet labs. Accidents are absolutely a possibility, and I don't think it's low given the previous behaviour of Silicon Valley startups.
* OK, some people care that AI is getting more capable at genetics just like it's been in maths and programming, but short term, before AI solves genetics so hard it can make an STD that makes the infected uncontrollably horny and then ossifies our bodies or something, it can obviously just copy any of the many DNA or RNA sequences we've already got on record.
Physical labs and experiments will always have physical bottlenecks, so I really don't see how AI is a transformative element here. What is it about AI that will be so transformative to the fundemental economics of this space that it will lead to several orders of magnitude more experimentation? If the problem is an expansion of wild west wet lab experimentation, why isn't ~that~ the focus of safety?
Who is going to profit from making an super STD anyway? How would regulating AI change their incentive structure?