I think the network effects make email hard to replace, virtually everyone online has an email address. If this could include a migration path, with backwards compatibility with SMTP, I think it would have a better shot of getting adoption.
Modern email already depends on HTTP, with protocols like MTA-STS (https://www.rfc-editor.org/info/rfc8461/) using HTTPS/TLS to improve transit encryption, or Web Key Directory (https://datatracker.ietf.org/doc/draft-koch-openpgp-webkey-s...) which uses HTTP for public key distribution. Personally I think these incremental improvements of SMTP are more promising than replacements, but we deserve better email however we can build it.
There is an easy way to mostly replace email over a decade, with the NewEmail protocol
NewEmailServers are able to send and receive messages using either the Email or the NewEmail protocol.
Before sending a message, the NewEmailServer checks a directory (similar to DNS) to see if any of the recipients are on another NewEmailServer. If so, the message is sent to those recipients via NewEmail. Others get it by Email.
Completely backward compatible. Once google/microsoft shift to NewEmail, the rest will follow quickly except some legacy servers. All that is needed is a standards body.
Basically what iOS messages does with SMS. I would note that this hasn't resulted in everyone using iOS messages as a communication protocol, but it does make users aware when it's not in use, with some inherent social pressure.
When dealing with any kind of network effect and existing user behavior, though, especially one as ingrained as email, I would highly recommend being judicious about using the word "just," given that these are mutually-reinforcing effects. Imagining a solution is a world away from implementing one.
Still, I think you're on the right track: create a new protocol on top of an old one, use the same input methods and user behaviors, call attention to when someone is not using it (e.g., green bubbles), and make it extremely easy for existing services to migrate to use it (i.e., the standard/standards body, notably NOT what Apple has done). One part that's missing here is some pressure to make this happen: positive pressure (economic, product sense, persuasion) or negative (economic, social, regulatory).
> we deserve better email however we can build it
Let's start with grouping messages into conversations reliably. Whatever black magic they are currently using is almost there, but has it's flaws.
The sad thing is that the only successful attempts at tackling its popularity are proprietary, closed ecosystems.
Yeah, this subject comes up every couple months. The problem is not getting it working. Its that email is a captured system at this point and the real work is various reputation management processes that the big providers control or you can't send email to/from them. They also have no real interest in letting you be your own email provider so its just pushing a rock up a hill grind, for what? To save <$100 a year piggybacking on another provider as a custom domain? or just use a free provider?
Having worked at a provider, that is not the psychology at all. The #1 priority is to keep the mail flowing. #2 incremental improvements to security, spam, reputation is a distant second. The other thing is, providers already know and have people who work on email standards, work in the ecosystem, etc. So once you're inside, with people who know whats going on, know the tools, know how things work intuitively without having to read a manual, theres less desire to change.
I have no personal experience. I've just read a lot of peoples experience around trying to do their own email hosting. It seems the majority give up after a couple years of nonstop blacklisting problems. It could simply be only people with issues post about it.
The stories have discouraged my from spending my time on it though.
As former Email Principal at major provider, sure, they run into blacklisting problems.
However, most of time, it's not that providers are blacklisting because "Ha ha ha, screw the little guy" but more "Yea, they are hosting on IP blocks with terrible reputation and spammers are absolute liars. I refuse to trust whatever they say, period."
If you have your own /24 or greater, you can become email provider. It's also just such a low margin business, it's not worth small businesses disrupting it.
I have found that if you harsh blacklist from most countries you'll never communicate from you'll save a lot of spam.
I blacklisted pretty much all of africa and arabic nations and china on the IP level, ( ensure you get the bgp routed datacenters in those nations too) and not only did i get less spam, but less automated bot attacks on my webserver.
The reason people talk about the blacklist is because its just so damn effective.