I wrote a guest post for High Scalability about how we scale our backend systems at VividCortex. It’s heavy on MySQL, sprinkled with a little bit of Redis’s magic pixie dust, and Kafka is also a key part of the architecture.
Important takeaways:
- Our workload is unusual in that some use cases require reading huge amounts of data with interactive response times. We have lots of time-series data, but not a typical time-series workload.
- Our data distribution is also weird.
- Read optimization is vital, but we are extremely write-heavy.
- We’re sharded and partitioned.
- We run our systems on just a few low-end EC2 machines.
- Go is a huge part of our speed to market and cost efficiency.
- We use a “just enough micro-” …