I think next time I upgrade my "phone" I'm going to get a gaming capable tablet with wireless and give it the steamos treatment. This gives you decent linux/windows/android interop.
I already lug a small backpack around most of the time, I can leave the tablet in the bag and use buds for conversations and when I need an actual computer it'll be way better.
Hm, how do you plan on taking calls? Will you notice an incoming call even though your tablet is in the bag?
(Asking because this idea sounds appealing to me as well.)
As is typical on Linux, lots of duct tape. Probably starting with https://github.com/dscripka/openWakeWord and speaking to https://github.com/rafael2k/ofono, then hacking from there.
Will you be able to notice when you receive a call? The only way that I can think of off the top of my head would be to make it ring out loud, which is more disruptive than a phone in-pocket set to vibrate.
My thought was a tone in buds (if they're in my ears) or just vibrate/straight to message otherwise, but I'm sure I'll be doing a lot of tinkering.