These concerns are exaggerated. I've been running on Hetzner, OVH and friends for 20 years. During that time I've had only two issues, one about 15 years ago when a PSU failed on one of the servers, and another a few years ago when an OVH data center caught fire and one of the servers went down. There have been no other hardware issues. YMMV.
They matter at scale, where 1% issues end up happening on a daily or weekly basis.
For a startup with one rack in each of two data centers, it’s probably fine. You’ll end up testing failover a bit more, but you’ll need that if you scale anyway.
If it’s for some back office thing that will never have any load, and must not permanently fail (eg payroll), maybe just slap it on an EC2 VM and enable off-site backup / ransomware protection.
Wasn't my product as a product manager but my long-ago company came out with an under the desk minicomputer product for distributed sites. And they didn't use ECC memory in the design. The servers didn't fail very often but multiply that fairly low error rate by a large number of servers and a system failure was happening every few days or so. The customer wasn't happy.