If those with assets keep accruing more assets the median person will suffer.

How will they suffer? The people with assets, to realize a benefit from them, have to spend money. If they don't spend the money, then what's the problem?