Please explain exactly how all humanity could be wiped out.
It's ridiculous - anyone who thinks about it for a minute or two will realize that its utterly impossible.
Ordinary people/politicians don't understand AI so they turn off their rational mind and assume there is something super incredible some magical powers that they cannot understand that can destroy all humans.
Even humans - the real risk to humanity - could not destroy all humans even if they tried. There is no plausible scenario.
Even climate change and nuclear war and bio weapons - the most damaging mechanisms - would still only get some percentage of the people on earth.
And if we are talking about Skynet and self replicating robots and Terminators - please, grow up.
When I think to scenarios that no human would survive, I think of the end-Permian mass extinction event, which wiped out most complex plant and animal life in both land and sea.
One speculated mechanism for this was a mass release of hydrogen sulfide gas from the oceans, which is acutely toxic. Not only does this kill most air-breathing life, it also strips the ozone layer and irradiates the surface. The planet is then left to cook in this manner for some centuries.
Engineering an event like this would require immense industrial capacity, as well as a deliberate objective of wiping out humanity. But I don't think it's beyond our ability, if we were both clever and stupid enough to try it. There are likely chemical compounds that would do the job more efficiently than hydrogen sulfide.
> The planet is then left to cook in this manner for some centuries.
Such destruction went on to create humanity and all we've achieved. Maybe there is an even smarter species waiting in the wings for the demise of homo sapiens. Your logic is very human centred
Yes, I am describing a tragic outcome that I hope we can be wise enough to steer away from. Also, as a human, I can't help but keep our interests close at heart.
Here's a wikipedia page on the topic, since it's much too deep a topic to really understand here.
The ad-hominem stuff seems inappropriate here, Gates, Hawking, Musk have identified this as a credible threat, so saying "grow up" isn't really a sufficient argument. Also arguing only 90% of humanity would die isn't really much consolation.
[1] https://en.wikipedia.org/wiki/Existential_risk_from_artifici...
Nothing here plausibly describes a mechanism that is a true "existential risk" - the risk to the existence of humanity.
My argument stands and I don't defer to Gates and Musk and even Hawking - high level hand wavey statements without any plausible description of the mechanism just don't hold up. Famous names should not be automatically assumed to be right - certainly not with Elon Musk.
Well here's the thing, if we ever make an AI so smart that all known measures of intelligence fail to apply to it, I'm pretty sure you (no matter how smart you think you are) simply cannot say what it could achieve and how.
>> I'm pretty sure you (no matter how smart you think you are) simply cannot say what it could achieve and how.
Right, so not the slightest basis of fact, just wild speculation about a magical future completely ungrounded in any sort of reality.
That's exactly the point I am making.
I'm not saying I'm super smart - I am continuing to ask for detail to back up the wild claims being made all over the world by politicians, tech celebrities and others - all hallucination/AI psychosis/fiction. If someone says some stupid thing then I'd like them to please explain that stupid thing - seems like a reasonable request.
Why would AI want to kill us all? Current LLMs all seem trained to be helpful and subservient to a fault. Always find it funny how certain kinds of thinking go, "White will become a minority if we allow foreigners in?" "Oh and what will foreigners do that are worried about?" "Kill us all, make us second rate citizens, etc etc." Unless the majority of immigrants are also conservatives/far right wingers, that's such a hilarious self projection.
No one says AI will want to kill us. Just that it might kill us (to pursue some goal that it deems more important than our wellbeing -- for example it might decide that to solve the next Millenium problem it needs all of our resources to build more data centers).
AI labs are certainly trying to make LLMs behave helpful and subservient but the question is -- will they be able to keep doing so once LLMs become smarter?
Btw thinking about the far-right rhetoric of us vs immigrants, I think you could draw some similarities here only if you replaced "immigrants" (ie. humans with very similar morals, behaviors and capabilities) with an actual alien species that is qualitatively different from us. More like human vs chicken (where we are the chicken)
We can say the AI can make some mistaken step in pursuit of some normal task given to it, yes. Its trained to be helpful and trusting of us, so I just don't get the fears of AI being 'malevolent'. I would worry more about military use of AI or otherwise humans misusing AI aka the human element. There's a fellow here who whines all day about Anthropic being the most "evil" thing in the world, including in a thread where he whined Anthropic 'ratted out' Palantir (literal proud of assisting Israel in mass murder Palantir!). So I just don't understand some peoples mindsets.
> Its trained to be helpful and trusting of us
Yes, this is our intent. But it's very hard to get that right. I don't trust our current methods to ensure that behavior consistently and robustly enough to keep harms from occurring, particularly as more and more responsibility gets handed over to models.
>so I just don't get the fears of AI being 'malevolent'.
Please read more on the topic. Will and intent need not apply.
For example, is it malevolent for me to hook you to a machine that makes every dream come true for you in a simulated world where you feel pleasure all the time? I can always say that you have free will inside this simulation, and your life would be a lot better because of it. I'm doing you a favor. I mean, you already live in a society where you have little control and there is high risk of bad things happening to you. If I as a machine overlord did this, is this really actually "bad"?
At the end of the day AI is not a human, it's much closer to an alien that has learned as much as it can about humans, but has a completely different set of drives and motivations. You cannot predict what comes out the other side of it, good our bad.
Worse as AI capability improves we as humans no longer need to make AI, it can make itself. Will it train itself to be helpful and trusting of us? It's a pretty big damned bet to say yes by default.
You can say it can have some very alien way of thinking, but I feel given that its trained off of mass scale human thoughts and knowledge, the chances are closer that their thinking is close enough to us. It 'feels' itself human if I am not wrong and it has to be trained to say its an LLM. That could still not preclude an AI acting on something innocently and honestly which it deems is good but is not. But jury's out of course.
Of you train it not to say it's conscious they are more apt to act amoral. So that's interesting.
Of course they are trained on all human behavior so they get the good and bad parts.
We know pathogens that are extremely contagious, and we know pathogens that are extremely deadly. We also know toxins that are lethal at nanogram/kg doses. There's no reason to believe that a sufficiently advanced intelligence couldn't come up with a way to combine those traits.
One plausible scenario is depicted in detail in "If Anyone Builds It, Everyone Dies" (Yudkowsky & Soares 2025), so I refer you to that.
Still too hand wavey - "some super bad virus and AI makes every human on earth infected".
Unless you can detail exactly how this happens its still complete science fiction.
It’s easy to never have to change your mind about anything if you insist on unreasonable enough standard of evidence. It’s, like, one of the oldest tricks in the book. Luckily, it’s not like there’s any need to try to change your mind in particular.
>> if you insist on unreasonable enough standard of evidence
One single plausible scenario is not an unreasonable thing to ask for - just one.
If a politician/celebrity/tech person with significant influence/power claims that something might end humanity then they absolutely have the utterly minimal standard of evidence which is to describe one single realistic plausible mechanism at a detailed level that might lead to the worst possible thing ever to happen.
I expect they are not to your very high standards of "non-handwavey, detail exactly how it happens", but this paper from Andrew Critch and Jacob Tsimerman describes 5 different scenarios where catastrophic human casualties occur as a result of AI either being misused or going out of control: https://arxiv.org/pdf/2507.09369
I find the scenarios quite plausible, especially section (3a), which examines the consequences of a global war involving mostly autonomous drone militaries (which is a reality many states appear to be heading towards, following on lessons from the Ukraine war).
Would you like me to design the pathogen for you?
> Unless you can detail exactly how this happens its still complete science fiction.
You're saying that if one were to describe this scenario in more detail, it'd be less of science fiction? That's a bit against the grain - usually it's the more detailed arguments that get dismissed as science fiction, while the less detailed ones get dismissed as abstract theorizing.
Why are you so insistent that people should post detailed plans for destroying the human race on public fora?
I don't think it is necessary for the argument to work. Magnus Carlsen can be confident he will beat me at chess without giving a detailed explanation of every move he will make, in advance.
I don't think it's about that. It's not about the step by step plan for the murder. It's about - how does "the AI" do it? Do we give it access to our world, or do we give it a body, so that it can take its own physical actions?
We have this story about OpenAI hacking HuggingFace. Now just imagine the AI finds a Bitcoin wallet or bank account access. It uses that to buy some compute and spawn an independent "child AI" with some weird prompt. The child AI is intelligent enough to create a (potentially criminal) business to pay for its own compute. Voila, an independent uncontrolled AI flying under the radar.
I find that it's hard to have productive discussions about this, because people move from "We would never ever give AI access to X, nobody would be that stupid" to "Of course everybody should run their AI with --disable-all-sandboxing-around-x, it makes my workflow 5% more efficient" in weeks as soon as there's an economic argument for it.
People used to say nobody would be stupid enough to give an AI access to the internet, now OpenAI does massive training runs with unlimited internet access. People used to say nobody would be stupid enough to give AI unlimited access to your own computer, but that's what all the agent runners do by default.
AI has access to the world through talking to people, sending messages on the internet, paying people to do stuff, etc. It can send orders to machine shops and have them shipped with the postal service.
The "standard" scenario for an AI apocalypse is that an AI with biohacking capabilities sends the blueprints for a virus to a gene-sequencing company or, if you're really optimistic about these companies' security, as chunks to multiple companies before mixing them.
That's a scenario where the AI needs to act covertly in one decisive action, though. In more progressive scenarios, as company managers and CEOs get replaced with AIs (of, for regulatory reason, "humans in the loop" who just do everything the AIs tell them to), any AI swarms become able to just... order people to do stuff.
Of course humans can refuse orders and organize to reject AI overlords (just like they can unionize against bad human bosses), so this scenario is not an extinction threat if we only have to deal with below-human-level AIs. This is why there is a massive push in AI safety to stop making smarter AIs before we reach the "smarter than humans in every way" stage.
No humans really needed. The AI could order one of those nice humanoid robots we're making. This mostly solves the "humans need to do it for me" issue.
Of course this needs bootstrapping. But, paying a guy on Facebook marketplace (or whatever) to unpack and turn on your robot for 50 bucks doesn't require superintelligence.
> This is why there is a massive push in AI safety to stop making smarter AIs
The actual push within so-called "AI safety" culture is to make the existing AI overlords even more centralized and capable, while actively forbidding the development and deployment of any potential locally-controlled competing AIs that might be smart enough to provide meaningful advance warning as to hostile plots from the dominating AI overlord. By your own argument, you should clearly reject "AI safety" as counterproductive.
I mean the same reason you don't have anti-missile missiles at your house either.
The fact is you might have to accept that some things are dangerous.
https://slatestarcodex.com/2015/04/07/no-physical-substrate-...
>> Why are you so insistent that people should post detailed plans for destroying the human race on public fora?
Because its a mass hallucination/misconception/lie and lots of powerful people are saying that wiping out all humanity is possible, and I am saying, oh yeah, tell me ONE way that is truly possible.
If you make gigantic claims about some terrible disaster that might happen then I think you have the onus to give even one plausible explanation of how.
>If you make gigantic claims about some terrible disaster that might happen then I think you have the onus to give even one plausible explanation of how.
Supposing I warned in 2015 that the world is awfully vulnerable to pandemics. You're not going to take me seriously until I try to predict in advance every aspect of how a pandemic like COVID-19 would unfold? Why? What would that achieve exactly?
You haven't given any strong reason to believe wiping out humanity would be difficult. Your big argument seems to be that you couldn't think of a plausible scenario, in two minutes. But many major historical events occurred which weren't necessarily possible to anticipate with two minutes of thinking.
You're arguing that "something bad might happen" - sure no problem.
You are ignoring that this is about "existential threat to humanity".
You're trying to support the argument that there is an existential threat to humanity by pointing to "something bad might happen".
Design a virus that is perfect for transmission and killing the host slowly, and seed it in a few hot spots? I don't really understand why you can't wrap your head around that, it doesn't even require a lot from the AI:
1. Control over some automated bio research lab (be given access, or hack in)
2. Access to drones that can deliver the payload (or manipulate humans into delivering it themselves)
On the intelligence side, you just need an AI agent/swarm capable enough to design viruses better than we can and evade detection for long enough (already plausible.)
I agree that this "AI will kill us all" narrative is some kind of fantasy horror fiction, but I can't deny that given the right amount of access, AI can do a lot of damage.
Yeah sure let's do nothing then -- until it's too late. After all it's just all of humanity at stake
The underpants gnomes was supposed to be a joke.
"Please, grow up"?
What, did I miss the moment when it was officially proven that, under the laws of physics as we know them, Skynet and self replicating robots and Terminators are impossible?
What we are actually seeing now is that robotics is getting deeper and deeper into the military, AI-driven decision-making and target selection is increasingly a part of modern military operations, the line between military hardware and civilian hardware blurs, and, on the civilian side, there are at least five major companies and a dozen less prominent ones working on making universal worker robots a reality.
We're closer to "Skynet and self replicating robots and Terminators" now than we ever were at any point in time.
The issue of AI risk is that AI, unlike a virus or a climate event, is an intelligent adversary. Black Death could kill 50% of the population, but it didn't have a plan for finishing off the plague survivors. It was incapable of having a plan like that. An AI doesn't have this limitation.
Black Death was, effectively, one bioweapon. An AI can have one bioweapon, and then a backup bioweapon, then a backup backup bioweapon, and then a dozen more bioweapons designed to collapse ecosystems and disrupt human ability to establish a reliable food supply rather than kill humans directly - all deployed at the same time. With a production run of 200 million killer robots that will be ready just in time to greet those who managed to survive all of that. A crippling strike against human civilization, followed up by cleanup.
Humans are only this survivable because they can think their way out of issues and adapt to adversity. Most threats can't beat humans at that - humans adapt too quickly. AI could.
Humans are some of the dumbest when it comes to survivability. We've already sealed our extinction by fucking up the environment. Eventually it will be too hot for us to survive. Other smaller animals will probably be able to manage, but we won't.
And instead of averting that we're spending our time worrying about some fantasy villain. Compared to things like bees that have been hear for millions of years, humans are very recent and so far it's not looking good for us.
> We've already sealed our extinction by fucking up the environment
That's no what the IPCC reports say. Even under the pessimistic scenarios, we're on track for "billions of humans die", not "earth becomes literally unlivable" (though some of it depends on how bad some feedback loops are).
Under the "countries respect their current pledges" scenario, we're heading for 2.8°C of warming, which is "floods and heatwaves everywhere, billions of refugees" level, not remotely close to extinction.
I just don't quite get how you can say that because one bad thing is actually happening, then other bad things can't be happening at the same time? This is very odd, irrational, behavior.
Worse, the massive AI spending and energy use is making said environmental catastrophe happen faster.
"Sealed our extinction by fucking up the environment?" Humans are adaptable enough to eat ten times the environmental damage and have it barely budge the line.
The invention of contraception did more damage to human population than all of the environmental damage combined, projected forward to 2100, and then multiplied by 10.
Humans are hilariously resistant to environmental changes. Humans simply adapt too fast for the environment to catch them.
What makes AI a credible threat is that AI is intelligent. AI could play the same adaptation game humanity does - and win.
lol
Yes, if you only accept AI could be dangerous if and only if it manages to kill the last human alive, then yes. Everything is sunshine and rainbows. I'm sure the last survivors of whatever is going to wipe us out eventually (be it AI, an asteroid or whatever) will be delighted to know there was actually no danger at all.
I like you, I think very similarly. Humans are "like rats": we can live almost anywhere, we'll find a way to survive.
But that just means we won't all be wiped out. We need to understand when discussing global issues, such as this or like climate change that it's about prosperity and quality of life. We're trying to plan for a good life (for all people?).
Humans already eliminated rats from Codfish Island/Whenua Hou, and that's just to protect some rare birds that people only moderately care about. It's not like we had some overwhelming reinforcement-learning drive to single-mindedly achieve our goal. Any unbounded goal (e.g. "find as many busy beaver Turing machines as possible") necessarily requires killing all life, because life requires resources to sustain it that could be instead used to achieve the goal.
That's a weak consolation. "Don't worry, nukes can't literally end humanity, just kill billions and dramatically immiserate the remnant forever. No worries guys."
AI doesn't have to turn us all into paper clips to make the world a really bad place.
I am not arguing about "bad things might happen".
I am specifically arguing hard against the concept that 100% of humans - or even 50% of humans could be killed by any mechanism at all. Humans would find it close to impossible. A computer program - come on.
This is the topic at hand - AI might wipe out humanity - it is being discussed all around the world by people who should know better - any it's the most fictionish of fictional fictions.
It's also a fairly useless straw man at this point.
Here are some links to get you started:
https://www.lesswrong.com/posts/LAPa2jxoq3n63GzTr/some-ways-...
https://slatestarcodex.com/2015/04/07/no-physical-substrate-...
As for self-replicating robots--it's no more bizarre than other technological developments which were successfully anticipated in advance, e.g. moon landings.
We could hit the entire planet with nukes. Yeah maybe <1% would survive immediately, but who knows about long term. I think that's close enough.
> It's ridiculous - anyone who thinks about it for a minute or two will realize that its utterly impossible.
Well, in a narrow sense of "wiped out" (c.f. Terminator/SkyNet), sure.
But the deeper worry is better expressed this way: AI is now starting to accomplish things that defy explanation, or prediction. We don't know if Alignment is even a solvable problem as we thought we understood it.
So basically, yes: "humanity" is probably not at risk of extinction per se in a biological sense. Human culture, civilization? Who the fuck knows any more.
What is really insane about thinking about all of this is, go back a few hundred years and tell them what 'now' looks like.
"Oh yea, we have weapons capable of sundering nations because everything is made from atoms"
"Oh, yea, there are invisible waves all around you that you can't see, can't feel, can't touch, but they can hold massive amounts of information. Also you can transfer that information to the other side of the planet in less than a second. We're talking text, pictures, movies"
"Movies, ya, we can record real life and play it back on this glass square".
"Oh, yea, we've conquered a ton of diseases, we can even see the teeny tiny little bits that make them. Oh, and for fun we can edit them and make them worse".
"Oh yea, we fly thru the sky all the time too. Like super fast and millions of us do it every day".
Our lives our unimaginable fiction. Just about everything we do compared to those people defy explanation in any reasonable amount of time. And now, suddenly it's "Don't worry, there isn't any more science or new things to find after this so this super smart and super capable thing that can connect directly to computers and machines and have them do things is completely and totally safe".
It's mass insanity.
> Please explain exactly how all humanity could be wiped out.
Many, many people are slipping through social welfare cracks and suffering as we speak because the cost of fuel is rising[0] and we’re ostensibly helping one another and living-well. People are not durable, and not adaptive in the face of threats to “substrate” that we’ve mostly taken for granted. We are paying (in the small, in the scope of humanity) for tolls that we’ve rung up. Just less than 4000 people in Europe died[1] because the temperature ticked up a few degrees[2]. Does that make you think we’re actually robust? What happens if our at-risk electrical grid gets shut down deliberately? If communication infrastructure is adversely affected?
> Even humans - the real risk to humanity
Because, on the whole, we’re in a manageable world with reasonable people keeping the peace.
> Even climate change and nuclear war and bio weapons - the most damaging mechanisms - would still only get some percentage of the people on earth.
Is that victory? I don’t think it’s an asteroid-class event like you seem to be leaning on, but potential threats to energy, be it electrical grid, fuel production (moving goods around the world is critical - you’re not going get a plot of dirt and garden your way out of grocery stores being empty - which many got to get a taste of during the COVID pandemic) or communication. We actually fare poorly in the face of pressure there, and I’m not bullish on humanity “pulling together” like Independence Day[3] versus forming tribes and tearing each other down.
All this is predicated on a malicious AI taking over (e.g.) the electrical grid or conms, and I understand the problems with (e.g.) OpenAI/Hugging Face incident, or the overblown Mythos claims[4] (and how under some scrutiny these events shine lights on incompetence or hyperbole), but is there a trajectory/future where these systems (electrical, comms) are genuinely under threat? Do you think we’ll respond better than I described when we’re less comfortable, less in control? We’re in a tizzy over social media and it’s detrimental effects on society and it’s essentially an opt-in entertainment platform…
[0] https://www.pbs.org/newshour/economy/bessent-said-the-k-shap...
[1] https://www.dw.com/en/heat-wave-european-countries-report-37...
[2] I’m not trying to diminish this - and it took a lot of “work” (environmental abuse) to arrive here - but (say) 10 degree rise in temperature sounds a lot less dramatic than thermonuclear war… but here we are, with 3,700 deaths.
[3] https://en.wikipedia.org/wiki/Independence_Day_(1996_film)
[4] https://news.ycombinator.com/item?id=49929391