This is not as simple as you're saying. Making a new phone not relying on proprietary drivers tied to Android is impossible without a huge effort: https://news.ycombinator.com/item?id=21656355

Obviously you'd only choose hardware that works the way you want it to.

Hardware relying on free drivers is almost non-existent in the mobile world. There is nothing to choose from, obviously.

Then aim for freely distributable drivers. You can share copies of Raspbian, so it seems possible.

You mean the Linux distro that exists because it needs to contain broadcom drivers/blobs/etc that are under NDA?

Then your hardware will turn into e-waste as soon as the vendor decides so and stops updating the drivers.