Higher supply still means lower prices. Even if "the wealthy" buy it all and rent it, they are still competing with all the other wealthy who are doing the same thing and that will limit how much they can charge for rent, and will make it a less profitable investment than other things the wealthy could invest in.

Housing is not really a great investment. It's great for small investors because it's the only place where they can invest with leverage via a mortgage. If you have billions there's much better things you can invest in.