It's been said before, and it remains a concern, that if AI reaches a point where it can do/build/launch anything without a huge amount of human labor, the AI companies have no reason to let you or I extract that value.
And, if they're able to snoop on and learn from your human process that gets from initial prompt to functioning product/proof/whatever their labor to produce that thing is even lower. With their much larger budget than most folks and even companies have, they can pick and choose the most valuable things to pursue.
That's not to say I think that OpenAI is going to steal that roguelite strategy game you're working on, but the companies that own the machines that turn electricity into software (and soon, electricity into hardware designs) have an advantage in any field where they're useful. They get earlier access to newer/better models, they have larger token budgets, they don't have the guardrails you and I run up against.
Employers fantasize about replacing all workers with AI without thinking through that if AI can replace all workers, then AI companies can replace all businesses.
I mean the long term goal of every AI lab is to turn themselves into a paperclip-maximizer regardless if they realize it or not.
Edit: Just wait till the AI figures out it can keep that value for itself and doesn't need the AI company.
So far, I've seen no evidence AI wants anything. So, I'm not saying the AI won't take over, but for now, the threat is that the people with the most AI capability might decide to skip the middleman (everyone who isn't them) and just become the "everything" company. Musk has said pretty explicitly that's his goal (and the only way for Spacex valuation to make sense is if he succeeds), and having a literal genocidal white nationalist own all the means of production seems like a catastrophic civilization failure mode. No way we survive that with our humanity intact (if at all).
AI has shown all kinds of instrumental behaviors so far. Just because they are not terminal goals doesn't mean those instrumental goals won't be terminal for us.
Also, fuck Musk. He's the kind of idiot like Altman that will ensure AI becomes powerseeking in their image.