Mar
27
2014
WebScaleSQL: A collaboration to build upon the MySQL upstream
Posted by MySQL@Facebook on Thu 27 Mar 2014 15:00 UTC

To help the more than 1.23 billion people who use Facebook to share and connect with each other, we’ve had to build an expansive and incredibly advanced infrastructure -- including one of the largest deployments of MySQL in the world. Along the way, we’ve learned and benefited from code changes made by the MySQL community. Today we’re announcing WebScaleSQL, a collaboration among engineers from several companies that face similar challenges in running MySQL at scale and seek greater performance from a database technology tailored to their needs.

WebScaleSQL currently includes contributions from MySQL engineering teams at Facebook, Google, LinkedIn, and Twitter. Together, we’re working to share a common base of code changes to the upstream MySQL branch that we can all use and that will be made available via open source. This collaboration will expand on existing efforts by the MySQL community, and we will continue to track the upstream branch that is the latest, production-ready release (currently MySQL 5.6).

Our goal in launching WebScaleSQL is to enable the scale-oriented members of the MySQL community to work more closely together in order to prioritize the aspects that are most important to us. We aim to create a more integrated system of knowledge-sharing to help companies leverage the great features already found in MySQL 5.6, while building and adding more features that are specific to deployments in large scale environments. In the last few months, engineers from all four companies have contributed code and provided feedback to each other to develop a new, more unified, and more collaborative branch of MySQL.

But as effective as this collaboration has been so far, we know we’re not the only ones who are trying to solve these particular challenges. So we will keep WebScaleSQL open as we go, to encourage others who have the scale and resources to customize MySQL to join in our efforts. And of course we will welcome input from anyone who wants to contribute, regardless of what they’re currently working on.

What we’ve built so far:

We want WebScaleSQL to be able to collaborate effectively and to move fast. To that end, we have set up a system for collaborating, reviewing code, and reporting bugs. For example, to introduce a code change, a WebScaleSQL engineer can propose a change. Then a WebScaleSQL engineer from another company will review the code and provide feedback. If both engineers agree the change makes sense and is functional, it

will be pushed into the WebScaleSQL branch for everyone to use. Beyond this, each organization may further customize WebScaleSQL to suit its own needs, just as we all do today.

This has already produced exciting results. Working together, the engineers involved in WebScaleSQL have made major changes to aid in the development of the new branch, including:



What we’re working on now:

After these initial accomplishments, we’ve started work on a number of other improvements to upstream MySQL. A few activities that Facebook’s WebScaleSQL team is currently working on:


What to expect in the future:

We will keep all our WebScaleSQL work open, to create a useful branch for others within the MySQL community who are focused on scale deployments. We’ll continue to follow the most up-to-date upstream version of MySQL. As long as the MySQL community releases continue, we are committed to remaining a branch – and not a fork – of MySQL.

We’re excited to expand our existing work on WebScaleSQL, and we think that this collaboration represents an opportunity for the scale-oriented members of the MySQL community to work together in a more efficient and transparent way that will benefit us all.

To learn more about how to get involved, visit: http://webscalesql.org/