You've probably all experienced it; another outage and the database is the root cause. Why are databases such a frequent cause of problems in most tech stacks? Why can't we seem to solve these problems industry-wide? Are database engineers and database admins just bad at their jobs?
Over my career as a database reliability engineer, I've come to a conclusion which I don't see repeated often:
All practical implementations have to balance the opposing concerns of correctness vs. performance & availability (this is kind of similar to CAP theorem , but not exactly the same). Perfect correctness with no data loss across geographic distances would result in a database which is too slow or too costly to be useful for most applications. And these constraints cannot be overcome because it's the physical bounds of reality which imposes …
[Read more]