I think fundamentally, the concern is misplaced. The fact you need to work for wealth is a convention of our constraints. The change in constraints would lead to other means of distribution. It's easy to see if someone who believes more productivity is good would not see making jobs obsolete a real problem. Thew would see us adapting to the new conditions in a relatively short while.
> The fact you need to work for wealth is a convention of our constraints
The current constraint is "you need to produce to have things".
If one company's AI takes all the jobs, and thus does all the producing-to-have-things, the constraint transforms into "you need that company's permission to have things".
Hence the top-level question.
The new conditions almost surely being like the old conditions: slavery, sexual exploitation, etc.
Those who are concerned is implying that any new distribution mechanism is not going to favour them.
And under the capitalist system, if nothing changes, the "new" distribution system is indeed not going to favour them - at best there would be some sort of UBI, and at worst you would be left to starve in the streets.
However, i cannot see how one can transition to a new system, and yet have the existing powers in the current system agree and not be disadvantaged.