So, yeah, I guess there's much confusion about what a 'managed database' actually is? Because for me, the table stakes are:
-Backups: the provider will push a full generic disaster-recovery backup of my database to an off-provider location at least daily, without the need for a maintenance window
-Optimization: index maintenance and storage optimization are performed automatically and transparently
-Multi-datacenter failover: my database will remain available even if part(s) of my provider are down, with a minimal data loss window (like, 30 seconds, 5 minutes, 15 minutes, depending on SLA and thus plan expenditure)
-Point-in-time backups are performed at an SLA-defined granularity and with a similar retention window, allowing me to access snapshots via a custom DSN, not affecting production access or performance in any way
-Slow-query analysis: notifying me of relevant performance bottlenecks before they bring down production
-Storage analysis: my plan allows for #GB of fast storage, #TB of slow storage: let me know when I'm forecast to run out of either in the next 3 billing cycles or so
Because, well, if anyone provides all of that for a monthly fee, the whole "self-hosting" argument goes out of the window quickly, right? And I say that as someone who absolutely adores self-hosting...
It's even worse when you start finding you're staffing specialized skills. You have the Postgres person, and they're not quite busy enough, but nobody else wants to do what they do. But then you have an issue while they're on vacation, and that's a problem. Now I have a critical service but with a bus factor problem. So now I staff two people who are now not very busy at all. One is a bit ambitious and is tired of being bored. So he's decided we need to implement something new in our Postgres to solve a problem we don't really have. Uh oh, it doesn't work so well, the two spend the next six months trying to work out the kinks with mixed success.
Slack is a necessary component in well functioning systems.
And rental/SaaS models often provide an extremely cost effective alternative to needing to have a lot of slack.
Corollary: rental/SaaS models provide that property in large part because their providers have lots of slack.
Of course! It should be included in the math when comparing in-housing Postgres vs using a managed service.
This would be a strange scenario because why would you keep these people employed? If someone doesn't want to do the job required, including servicing Postgres, then they wouldn't be with me any longer, I'll find someone who does.
No doubt. Reading this thread leads me to believe that almost no one wants to take responsibility for anything anymore, even hiring the right people. Why even hire someone who isn't going to take responsibility for their work and be part of a team? If an org is worried about the "bus factor" they are probably not hiring the right people and/or the org management has poor team building skills.
Exactly, I just don't understand the grandparent's point, why have a "Postgres person" at all? I hire an engineer who should be able to do it all, no wonder there's been a proliferation of full stack engineers over specialized ones.
And especially having worked in startups, I was expected to do many different things, from fixing infrastructure code one day to writing frontend code the next. If you're in a bigger company, maybe it's understandable to be specialized, but especially if you're at a company with only a few people, you must be willing to do the job, whatever it is.
Because working now at what used to be startup size, not having X Person leads to really bad technical debt problems as that person Handling X was not really skilled enough to be doing so but it was illusion of success. Those technical debt problems are causing us massive issues now and costing the business real money.
IMO, the reason to self-host your database is latency.
Yes, I'd say backups and analysis are table stakes for hiring it, and multi-datacenter failover is a relevant nice to have. But the reason to do it yourself is because it's literally impossible to get anything as good as you can build in somebody's else computer.
Yup, often orders of magnitude better.
If you set it up right, you can automate all this as well by self hosting. There is really nothing special about automating backups or multi region fail over.
But then you have to check that these mechanisms work regularly and manually
One thing I learned working in the industry, you have to check them when you're using AWS too.
Really? You're saying RDS backups can't be trusted?
Trusted in what sense, that they'll always work perfectly 100% of the time? No, therefore one must still check them from time to time, and it's really no different when self hosting, again, if you do it correctly.
What are some common ways that RDS backups fail to be restored?
Why are you asking me this? Are you trying to test whether I've actually used RDS before? I'm sure a quick search will find you the answer to your question.
No backup strategy can be blindly trusted. You must verify it, and also test that restores actually work.
Self-host things the boss won't call at 3 AM about: logs, traces, exceptions, internal apps, analytics. Don't self-host the database or major services.
Depending on your industry, logs can be very serious business.
Yugabyte open source covers a lot of this
Which providers do all of that?
I don't know which don't?
The default I've used on Amazon and GCP both do (RDS, Cloud SQL)
GCP Alloy DB
There should be no data loss window with a hosted database
Feom what I remember if AWS loses your data they are basically give you some credits and that's it.
That requires synchronous replication, which reduces availability and performance.
Why is that?