Over the past few months, I’ve seen an increase in the following use case while working on performance and schema review engagements:

I need to store exponentially increasing amounts of data and analyze all of it in real-time.

This is also known simply as: “We have big data.” Typically, this data is used for user interaction analysis, ad tracking, or other common click stream applications. However, it can also be seen in threat assessment (ddos mitigation, etc), financial forecasting, and other applications as well. While MySQL (and other OLTP systems) can handle this to a degree, it is by no means a forte. Some of the pain points include:

• Cost of rapidly increasing, expensive disk storage (OLTP disks need to be fast == $$)
• Performance decrease as the data size increases …

September 18, 2014 By Severalnines

Galera Cluster is a popular choice for achieving high availability using synchronous replication. Though if you are planning to run huge sites with many DB objects (tables), a few tweaks are necessary. 

Yes, you might have been successful in loading your 1000s of databases and 1000s of tables, but what happens if you have a node failure and Galera recovery fails?

In this blog post we will show you how to determine one common error related to the open_files_limit that MySQL imposes, and also to spot another potential pitfall.

Open_files_limit

If you are using wsrep_sst_method=xtrabackup or wsrep_sst_method=xtrabackup-v2 then you will find a log file in the data directory of the donor node. This log file is called innobackup.backup.log.

140912 19:10:15  innobackupex: Done. …

Are you overwhelmed with the hundreds of MySQL settings? Lost trying to tune the query optimizer and InnoDB buffer pools? Is the configuration process daunting?

We are here to help. Download your free copy of our latest eBook, Configuring MySQL for Performance. In just minutes, you will gain insight into setting configurations for optimal MySQL performance.

It is fast, and it is free. Request your copy here.

Wed, 2014-09-17 19:03maxmether

Tonight I will give a presentation on Max Scale (An Advanced Proxy for MySQL) for the Chicago MySQL Meetup

MaxScale is an interesting new proxy for MySQL and MariaDB that provides advanced features for handling multiple MySQL or MariaDB servers. MaxScale allows you to transparently handle clusters of MySQL servers behind the proxy. MaxScale is different from most other proxies as it is content aware and contains a query parser, has a modular plugin architecture, is aware of the state of the underlying servers and much more. In this session we will look at the architecture of MaxScale to find out what makes MaxScale so good. We will look at the basic building blocks of MaxScale and how they work together in a very efficient way. We will also discover the basic modules delivered with MaxScale, such as the read-write …

I recently encountered a situation in which after running Percona Toolkit’s pt-table-checksum on a customer system, 95% of the table on the MySQL master was different on the MySQL slave. Although this table was not a critical part of the infrastructure, from time to time, writes to the table from the master would break replication. Additionally, this table has about 6 million rows, and running pt-table-sync would take sometime. Rebuilding the slave from backup of the master would not be an easy option as well since the slave acts as an archive where it has a lot more data than the master.

So how did we solve it? With pt-online-schema-change and a NOOP ALTER.

pt-online-schema-change --alter 'ENGINE=INNODB' D=dbname,t=tblname

How is it …

September 17, 2014 By Severalnines

The Severalnines team is pleased to announce the release of ClusterControl 1.2.8. This release contains key new features along with performance improvements and bug fixes. We have outlined some of the key new features below. 

Highlights of ClusterControl 1.2.8 include:

• YUM/APT repositories for ClusterControl
• Deployment and scaling of single-node MySQL, MariaDB and MongoDB
• Alerts and incident tracking with PagerDuty 
• Unified Event Viewer
• New flexible alarms/email notification system
• Audit logging - Administrator activity tracking
• Global MySQL User Management
• New default dashboards for MySQL/MariaDB
• Puppet Module …

The countdown is on, and MySQL Central begins in less than two weeks. We are pumped to be participating and want to make the most of this great experience. Here are five ways we plan to do so:

1) Meet the MySQL development team. They will be leading two sessions on Tuesday evening, both at 6 PM (BOF6883 and BOF5939, respectively). The first is on InnoDB, the MySQL optimizer, and any other MySQL server topic. The other discusses high availability and backup, including MySQL replication, MySQL utilities, MySQL Cluster, MySQL Fabric and MySQL backup. Bring your questions and ideas to the experts!

2) Plan your schedule in advance. Besides the two BOF sessions mentioned above, there are plenty of opportunities to engage with the community, participate in hands-on labs and learn from expert tutorials. …

It's that time of the year again: yes, Oracle Open World is coming up and with that I'll be travelling to San Francisco. New for this year is that we are part of the main Open World event and therefore have our own MySQL Central. Here you will have the opportunity of meeting many of the engineers behind MySQL, discuss technical problems you have, and also learn some about how we look at the future of the MySQL ecosystem.

This year, me and Narayanan Venkateswaran will be presenting two sessions:

Elastic Scalability in MySQL Fabric with OpenStack (Thursday, Oct 2, 1:15 PM-2:00 PM in Moscone South, 252)

In this session you will see …

As the amount of data companies are interested in collecting grows, life becomes all the more difficult for IT staff at all levels within an organization. SAS Enterprise storage devices that were once considered giants are now being phased out in favor of SSD Arrays with features such as de-duplication, tape storage has pretty much been abandoned and the same goes without saying for database engines.

For many customers just storing data is not enough because of the CAPEX and OPEX that is involved, smarter ways of storing the same data are required and since databases generally account for the greatest portion of storage requirements across an application stack. Lately they are used not only for storing data but also for storing logs in many cases. IT managers, developers and system administrators very often turn to the DBA and pose the time old question “is there a way we can cut down on the space the database is taking …

Every document I create these days is written in rst (ReStructuredText) and transformed into something useful using rst2pdf. This includes worksheets, reports, handouts and slide decks. Along the way I've learned a few tricks, and I try to write them down so I can look up how to do something. If this helps you too, then great :)

Start Simple

Include a code block in rst by adding something like this:


.. code-block:: php


Your code must then be indented, and the style persists until you "undent" again. There is pretty extensive language/syntax support since it is built on pygments, so you can use most language names instead of the php there.

Line Numbers and PHP Tags

In order to make the PHP syntax highlight …