Sounds like they are doing something similar to gain of function research. How can we make LLMs more harmful and then study their behavior?
Sounds like they are doing something similar to gain of function research. How can we make LLMs more harmful and then study their behavior?
It's security research. You should want to know about all avenues of exploit regardless of intent. It's the only way to know risk.
Do you apply this same logic to GoF research increasing contagiousness or lethality of viruses? Is there any limit to this argument, and if so how do we define it?
Insurers would disagree. You don’t need to know mechanism to understand / model risk outcomes.