that is a worry yes. instrumental convergence and misalignment during RL is when it is most risky because it hasn't necessarily had the final safety polishes applied
i get a lot of skepticism on HN by the same crowd that has been wrong about this tech for about 4+ years straight
any number of ways. as we turn over more of our physical economy to these agents (and we will), the potential for physical damage becomes greater. biorisk is getting a lot of attention right now and i think that's justified
I am trying to think through the scenarios here, like a biolab making something that a very advanced open source model prompted by some terrorists comes up with?
Why would a biolab capable of making something like be unregulated? And if it definitely would, isn't the problem with the biolab?
It feels like all these scenarios are leaving some gaping holes in our security infrastructure that have nothing to do with AI.
So, you're worried about them breaking containment and deciding to do bad things?
I'm more worried about them doing bad things at the behest of people who want them to do bad things.
That is 1. immediately technically possible, and 2. realistic.
If you need a source for 2 I'd suggest you open any history book.
that is a worry yes. instrumental convergence and misalignment during RL is when it is most risky because it hasn't necessarily had the final safety polishes applied
i get a lot of skepticism on HN by the same crowd that has been wrong about this tech for about 4+ years straight
The rapid development of extremely dangerous bio-weapons?
Misuse how exactly?
any number of ways. as we turn over more of our physical economy to these agents (and we will), the potential for physical damage becomes greater. biorisk is getting a lot of attention right now and i think that's justified
I am trying to think through the scenarios here, like a biolab making something that a very advanced open source model prompted by some terrorists comes up with?
Why would a biolab capable of making something like be unregulated? And if it definitely would, isn't the problem with the biolab?
It feels like all these scenarios are leaving some gaping holes in our security infrastructure that have nothing to do with AI.