> to pose existential risk to all of humanity

The defense againt rogue-AI is very simple. Go to the nearest data-center with a fire-truck, and hose it down. So, I am in the least bit worried about this.

Try this thought experiment: If someone took away AI/LLMs for good tomorrow would you miss it ? I can bet that the average person cares more for their smartphone/amazon/netflix than chatgpt. I consider LLM to be a useful tool but I would not miss it if you took it away from me. This should tell you its value to society and the economy.

I think engineers at the AI labs are too far into the reality-as-a-simulation rabbit hole. This can lead to these sorts of collective delusions. When you are in the real-world smelling the roses and grounded by your own qualia, such fears dont arise. Therapy helps here - to get out of the simulation and into the real world.