even if the most powerful ai is open source and let's even assume runs on consumer hardware - in the end data is the real moat.

if it can access private data it will necessarily have more power.

Let's abstract this further: It's about the user's existing power and intentions, meaning if I am already in a position of power, AI will multiply it to levels way beyond a peasant could. Power dynamics just get exacerbated.