When you develop an application you expect success, and often success comes with growth problems. These problems especially show themselves in the area of data storage, where being stateful is not as easy to scale as the stateless parts of the application.
There are several stages of approaching database scalability:
- Configuration and query optimization. This step can help a lot, and I would recommend a recent book by Daniel Nichter “Efficient MySQL Performance: Best Practices and Techniques”, which goes into this topic.
- If #1 is done and you continue to push the limits of your database, the next step is to improve the hardware: adding extra memory, improving storage throughput (regular SSD, NVMe storage layer, etc.), or increasing the size of the cloud instances (this is what I call “optimization by credit card”). This typically should help, but only to a certain limit. And there is only so much …