At Pinterest we’re building the world’s most comprehensive discovery engine, and part of achieving a highly personalized, relevant and fast service is running thousands of jobs on our Hadoop/Spark cluster. To feed the data for computation, we need to ingest a large volume of raw data from online data sources such as MySQL, Kafka and Redis. We’ve previously covered our logging pipeline and moving Kafka data onto S3. Here we’ll share lessons learned in moving data at scale from MySQL to S3, and our journey in implementing Tracker, a database ingestion system to move content at massive scale.

History

To give an idea of the challenge, let’s first look at where we were coming from. MySQL is the main data source for storing the most important objects in Pinterest: Pins, Pinners and boards. Every day we collect more than 100 terabytes …

Welcome to this week’s Planets9s, covering all the latest resources and technologies we create around automation and management of open source database infrastructures.

From October 3rd to 5th, the MySQL Engineering and Community Team will be in Amsterdam to present MySQL 8.0 at Percona Live.

The Conference Committee has rated all the numerous submissions and I’m very proud to announce that we will deliver the following 11 sessions:

• MySQL Group Replication in a nutshell: hands-on tutorial, by myself and Kenny Gryp, 3 October 1:30PM – 4:30PM
• MySQL 8.0: Overview, Geir Høydalsvik, 4 October 11:20AM – …

In this blog post, we’ll discuss how a small innodb_page_size can create a performance boost for SSD.

In my previous post Testing Samsung storage in tpcc-mysql benchmark of Percona Server I compared different Samsung devices. Most solid state drives (SSDs) use 4KiB as an internal page size, and the InnoDB default page size is 16KiB. I wondered how using a different innodb_page_size might affect the overall performance.

Fortunately, MySQL 5.7 comes with the option innodb_page_size, so you can set different InnoDB page sizes than the standard 16KiB. This option is still quite inconvenient to use, however. You can’t change innodb_page_size for the existing database. Instead, you need to create a brand new database with a …

A MariaDB support customer recently upgraded to MariaDB 10.1, and they noticed that some of their queries using bitwise operators started to return warnings, which they thought was strange because they produced no warnings in MariaDB 10.0. These particular queries used bitwise operators on BINARY(N) fields. For example, their table was similar to this: CREATE TABLE item_flags ( item_id int(11) ... Read More

The MariaDB project is pleased to announce the immediate availability of MariaDB 5.5.51, MariaDB Connector/J 1.5.0 RC, and MariaDB Connector/C 2.3.1. See the release notes and changelogs for details on these releases. Download MariaDB 5.5.51 Release Notes Changelog What is MariaDB 5.5? MariaDB APT and YUM Repository Configuration Generator Download MariaDB Connector/J 1.5.0 RC Release […]

The post MariaDB 5.5.51 and updated connectors now available appeared first on MariaDB.org.

The user camp has always been a great platform for the interaction between the user community and the MySQL team. We hosted the 11th User Camp on 29th of July 2016 at ORACLE office located at Bangalore. The audience was a good mix of DBAs, freelancers, MySQL enthusiasts, developers from other firms and also the members of the MySQL team.…

As announced by Luis Soares on MySQL High Availability’s blog, a new labs version of Group Replication (GR) has been released for MySQL 5.7.14 !

Group Replication is the main component of the future MySQL HA solution. Even if with GR, it’s possible to write simultaneously on all the member of the group, GR doesn’t provide any write scaling. Indeed, if one member reached 100% of its write capabilities, adding writes to another member, won’t help as both will have to write their load AND the load from the other member: remember this is (virtual) synchronous replication, all writes happening on member A will happen on member B and on all other members of the group.

As you …

In this blog post, I’ll discuss changes I’ve made to the

tpcc-mysql

 benchmark tool. These changes make it less random and support multi-schema.

This post might only be interesting to performance researchers. The

tpcc-mysql

 benchmark to is what I use to test different hardware (as an example, see my previous post: https://www.percona.com/blog/2016/07/26/testing-samsung-storage-in-tpcc-mysql-benchmark-percona-server/).

The first change is support for multiple schemas, rather than just one schema. Supporting only one schema creates too much internal locking in MySQL on the same rows or the same index. Locking is fine …

Join Sveta Smirnova Thursday, August 11 at 10 am PDT (UTC-7) for a webinar on InnoDB Troubleshooting.

InnoDB is one of the most popular database engines. This general-purpose storage engine is widely used, has been MySQL’s default engine since version 5.6, and holds MySQL system tables since 5.7. It is hard to find a MySQL installation that doesn’t have at least one InnoDB table.

InnoDB is not a simple engine. It has its own locks, transactions, log files, monitoring, options and more. It is also under active development. Some of the latest features introduced in 5.6 are read-only transactions and multiple buffer pools (which now can persist on the disk between restarts). In 5.7, InnoDB …