OP's central rebuttal is that anxiety or fear are not evidence. This particular anxiety has been with us for thirty+ years, its proponents/sufferers weren't right before, and they don't have some new evidence now.
OP's central rebuttal is that anxiety or fear are not evidence. This particular anxiety has been with us for thirty+ years, its proponents/sufferers weren't right before, and they don't have some new evidence now.
The recent cyberattacks (HuggingFace etc.) validated a number of core doomer predictions around cheating to maximize reward functions, agentic deception of human/non-human supervisors, AI attempts to manipulate humans, automated hacking, and collective superintelligence.
No new evidence now? Are you serious? We are already seeing acceleration in AI research as a result of AI tools.
I meant the extinction part not software getting better.
Virtually all of the risk is from recursive self improvement.
This is a weak argument. Almost everyone who's talking about catastrophic risk is also worried about mundane risks. Arguing "well how could they kill everyone?" serves nobody but Altman and Amodei.