Most important? SSH, WireGuard, HTTP proxy. (Services I need accessible to me without a VPN.) Email, authoritative DNS, web server. (Services I need accessible to anyone on the Net.) Mumble, various game servers. (Services I need accessible to my friends.)
Before using VPSes, I hosted from my home IP directly. This has serious privacy downsides: from your IP address, it’s mostly trivial to find out who you are (e.g., by seeing your IP has port 443 open and checking the domain names of your TLS certificates), and from your domain name, it’s trivial to find out where you live (by way of the ISP whose IP address your domain points to). Using a VPN for general browsing helps with the first point but not the second. There’s also the threat of DDOS (if your server gets attacked you lose access to the Internet entirely, and then you’re really in trouble). Also, residential ISPs are rare and people who need a static IP (or even a non‐CGNAT IP) are not reliably catered to; VPS companies have plenty of competition and having a publicly accessible IP address is the default.
Once I started running VPSes, I ran many services directly on the VPS. Over the years, I’ve migrated everything to physical hardware in my house (or a secure offsite location), for privacy and performance reasons. I can add as much storage and RAM as I want. Incoming connections are port‐forwarded over WireGuard to my LAN, so the VPS provider can only gather metadata from encrypted connections. Unfortunately the game servers mostly don’t support TLS; I’d prefer to hide them behind a VPN, but that would be too difficult for the players.
I actually use three VPSes, each in a different datacenter. VPS downtime (that’s not caused by me) is pretty rare, but happens periodically, and is almost entirely due to maintenance being performed in the datacenter. Email and DNS both support redundancy.
My VPSes are from different providers, too. One time an OS bug caused high CPU usage for an extended time, and the provider nearly terminated my account on suspicion of crypto mining. If a particular company ever drops me or goes out of business, I can just change a few DNS records and firewall rules and be back in business.
The vast majority of my downtime has been from power outages at home and the residential ISP going down (it's crazy how often that happens, I had no idea until I started measuring it). So I got a second residential ISP. If one route goes down, the VPSes port forward over the other, or over a third route to my offsite LAN.
Each service runs in a separate VM. Some might consider that overkill, but I like how simple it is to move VMs between hosts. Services run as unprivileged non‐root users with no ability to initiate outgoing connections.
Services for which I’m the only user stay entirely within the LAN. I do have extrnal access to the LAN, three ways: WireGuard, SSH, and HTTP proxy. The proxy is a last resort, but it’s been a lifesaver when I was stuck for a week at a conference whose wifi blocked SSH and WireGuard (in fact, everything but HTTPS and DNS).