Will second this, as someone that took the same path. Alternatives like Fastmail are so, so cheap when compared to the headache of building, maintaining, and troubleshooting a DIY stack. For something as fundamental as email you really want to choose the solution that gives you NO problems, rather than some solvable but troublesome problems at regular intervals.

Just too critical of a component in most people's lives to accept any level of unknowns in.