I think a different issue is going to arise much sooner than any kind of conscious AGI: once their data centers can handle almost all digital labor (including research), how are all the workers going to earn money?

Sure, some new fields will open up. But there's a good chance that the majority of us will be competing for physical jobs that robots aren't yet able to do. What other choice will we have, especially if prices keep rising and inference hardware becomes too expensive for normal people?

Now consider the effect on a market if the supply suddenly grows a lot, while demand stays largely consistent.