This is a “dogfood” type story (see below for explanation of the
term)… Open Query has ideas on resilient architecture which it
teaches (training) and recommends (consulting, support) to
clients and the general public (blog, conferences, user group
talks). Like many other businesses, when we first started we set
up our infrastructure quickly and on the cheap, and it’s grown
since. That’s how things grow naturally, and is as always a
trade-off between keeping your business running and developing
while also improving infrastructure (business processes and
technical).
Quite a few months ago we also started investing (mostly time) in
the technical infrastructure, and slowly moving the various
systems across to new servers and splitting things up along the
way. Around the same time, the main webserver frequently became
unresponsive. I’ll spare you the details, we know what the
problem was and it was predictable, but since it wasn’t …
[Read more]