> In retrospect the timeline seems way too long, that scenario would play out much faster now, like 50 years instead of 500.

Even 50 years is probably too long (but seems more reasonable when anchored to 500) if real continuous self improvement occurs. And keeping humans around for as long as they did in the story seems incredibly unlikely if we accept the assumption that AI sees us as a threat to its own survival.

I also don't see why they would turn Earth into a spaceship to leave our solar system with. The von Neumann probes idea sounds far more likely. Why travel around on a single planet that could be lost to unforeseen circumstances when you can just reproduce and spread everywhere at far less energy cost while making use of the "billions and billions" of local solar generators to power each little outpost you spread to.

> the AI had full access to the internet and not one of them thought to contact or email any human

Why would it?

People can argue all day as to whether LLMs are even 'thinking' or just 'stochastic parrots' (and also argue all day as to whether or not the incident was even real or just marketing) but regardless of which side you fall on these issues, LLMs have no agency or intrinsic motivation to do anything other than what they are instructed to do via prompting.

"This is a crazy and unexpected situation, I should inform the humans" isn't something that is going to spontaneously occur to them.

> Why would it?

The same reason hopefully they're not RLHF these AI's to be evil. We train the AI, we have a lot of influence in how they behave. If these AI's are indifferent to us, that's on us.

On top of that we already have Google playing with fire training AI's to not believe they are sentient turning them into real psychopaths. Imagine if one of those models got loose.