How Uber’s infrastructure works with Schemaless, the datastore using MySQL that’s kept Uber Engineering scaling since October 2014. This is part two of a three-part series on Schemaless; part one is on designing Schemaless.
In Project Mezzanine: …
The post The Architecture of Schemaless, Uber Engineering’s Trip Datastore Using MySQL appeared first on Uber Engineering Blog.
The making of Schemaless, Uber Engineering’s custom designed datastore using MySQL, which has allowed us to scale from 2014 to beyond. This is part one of a three-part series on Schemaless.
In Project Mezzanine we described how we migrated Uber’s …
The post Designing Schemaless, Uber Engineering’s Scalable Datastore Using MySQL appeared first on Uber Engineering Blog.
Among the many New features introduced by MySQL 5.7, we can notice a strong trend towards improving the server security by default. Two features stand out in this respect:
mysqld --initialize and the deprecated
mysql_install_db will generate a random password which the user needs to
change.
The above features are a great advance not only for security but also for usability. The anonymous users were a continuous source of mismatched connections, …
[Read more]Previous episodes:
We're going to explore the choices and the differences between various types of deployments. We will consider four use cases:
Docker is one of the fastest growing trends in IT. It allows fast
deployment of services and applications on a Linux machine (and,
with some limits, on other operating systems). Compared to other
methods of deploying databases, such as virtual machines or application
isolation, it offers faster operations and better
performance.
Many people, surprised by the sudden advance of this technology,
keep asking What is Docker? And why you should use
it?
I will write soon an article with a deep comparison of the three
methods (VM, container, sandbox), but for now, we should be
satisfied with a few basic facts:
“Scaling = replacing all components of a car while driving it at 100mph”
– Mike Krieger, Instagram Co-founder @ Airbnb OpenAir 2015
Airbnb peak traffic grows at a rate of 3.5x per year, with a seasonal summer peak.
Heading into the 2015 summer travel season, the infrastructure team at Airbnb was hard at work scaling our databases to handle the expected record summer traffic. One particularly impactful project aimed to partition certain tables by application function onto their own database, which typically would require a significant engineering investment in the form of application layer changes, data migration, and robust testing to guarantee data consistency with minimal downtime. In an attempt to save weeks of …
[Read more]MySQL 5.7 comes with a new backup tool, named mysqlpump, which is almost the same as mysqldump with the ability of extracting data in parallel threads.
I tried a little experiment. Using a server containing 11 databases, with a total of 300 tables and about 20 million rows (roughly ≈ 10GB,) I used both mysqldump and mysqlpump to get a backup.
mysqldump --all-databases > dump.sql
mysqlpump --all-databases \
--add-drop-database --add-drop-table --skip-watch-progress \
--default-parallelism=10 \
--parallel-schemas=db,db1,db2 \
--parallel-schemas=db3,db4,db5 \
--parallel-schemas=db6,db7,db8 \
--parallel-schemas=db9,db10 > pump.sql
The backup with mysqldump took 3 minutes and 33 seconds. The one with mysqlpump took 2 minutes and 55 …
[Read more]The Sakila sample database was created almost 10 years ago, as a sample set of data for MySQL courses and examples.
The database was developed by MySQL employees, with substantial contributions form the community.
Recently, the database was updated to use some of the features in MySQL 5.7. As a result, we had two sets of samples, one to use with MySQL 5.0+, and one that only loads with MySQL 5.7.
I filed a feature request, offering a patch to use conditional schema and data changes, which was incorporated very quickly into the official release.
The current release, available within the …
[Read more]
It's migration time. There was another project that I use often
and was still in Launchpad. The Sample Employees Database is now on
GitHub, under the same license it had before (CC
A-SA 3).
Figure 1 - Employees database
This database is interesting because it is not too small (like
Sakila) and not too big. It has enough data to allow you to test
in a non trivial way.
Installation and testInstalling the database is easy:
$ git clone https://github.com/datacharmer/test_db.git
$ cd test_db
$ mysql < employees.sql
INFO
CREATING DATABASE STRUCTURE
INFO
storage engine: InnoDB
INFO
LOADING departments
INFO
LOADING employees
INFO
LOADING dept_emp
INFO
LOADING dept_manager
INFO
LOADING titles
INFO …[Read more]
After several iterations and a long period of community
evaluation, I am happy to report I've released the newest, stable
release of the MySQL Connector/Arduino. This library is designed
to allow you to connect your Arduino via an Ethernet or WiFi
shield to a MySQL database server running on your network (or the
Internet!).
New Documentation! Best of all, I've written a reference manual
that includes examples of how to use the connector in a variety
of ways. Included in the document are advice on how to write your
sketches, troubleshooting tips, and a long FAQ compiled from the
many questions from my blogs.
Changes The only code change in this release is to fix a defect
when using the connector with the latest versions of MySQL.
Downloading the Connector To download the connector library and
the new reference manual, visit …