It is a common database pattern to increment an INT column when an event happens, such as a download. Early last year at GitHub we started encountering issues with slow UPDATES to these types of counters.

You can go far with this pattern until bursts of these types of events happen in parallel and you experience contention on a single row. When multiple transactions are trying to update the counter you are essentially forcing these transactions to run serially which is bad for concurrency.

You can see here a dramatic increase in query time when a burst like this occurred:

<img src=”/images/burst.png” height=“200px” width=”725px” />

To avoid problems like this we had to do this kind of counting differently. We decided on using a separate table with a schema similar to this:

CREATE TABLE `slotted_counters` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `record_type` int(11) NOT NULL, …

As a MySQL DBA/consultant, it is part of my job to decode the MySQL binary logs – and there are a number of reasons for doing that. In this post, I’ll explain how you can get the important information about your write workload using MySQL row-based binary logs and a simple awk script.

First, it is important to understand that row-based binary logs contain the actual changes done by a query. For example, if I run a delete query against a table, the binary log will contain the rows that were deleted. MySQL provides the mysqlbinlog utility to decode the events stored in MySQL binary logs. You can read more about mysqlbinlog in detail in the reference manual here.

The following example illustrates how mysqlbinlog displays row events that specify data modifications. These correspond to events with the WRITE_ROWS_EVENT, UPDATE_ROWS_EVENT, …

Apache CloudStack is a turnkey orchestration platform to deliver IaaS clouds. As a cloud infrastructure scales to hundreds or thousands of servers, with production environments supporting multiple applications and services, high availability becomes a key requirement. Just as for OpenStack, it is possible to deploy the CloudStack management server in a multi-node installation. Since the management server uses a MySQL database to store the state of all its objects, the database could become a single point of failure. The CloudStack manual recommends MySQL replication with manual failover in the event of database loss. We believe there is a better way.

In this blog post, we’ll show you how to deploy redundant CloudStack management servers with …

PayLease is the leading online payment processor for property managers and HOAs. The company serves thousands of property management clients across the United States and was seeking a solution to help them better manage their massive amount of data and minimize slow periods.

Before VividCortex, they had to sort through nearly 10 gigabytes of data each time there was an issue, a cumbersome process that often proved ineffective. PayLease also wanted a tool that would better equip them to prevent future issues. They needed a predictive solution to flag issues that could become high priority down the road.

PayLease chose VividCortex because of the intuitive interface, level of granularity and functionality. They saw the difference immediately. Todd Newell, the Database Engineer, “I look at top queries on a daily basis. I recognize what is there, what should be there, and am able to quickly pull something out that is garnering way …

Recently, we create a mysql data warehouse which is based on message queue.

Most companies must prepare for particular queries in their systems if they consider to split their databases or tables into many pieces.

some problems should be solved in this situation:

1. how to get correct results in-time
2. how to build strong data warehouse for future analyst

These policies were used by YHD

They have already deployed a middle-ware layer to support these requests (between web apps and databases). Every aggregation SQL was splited into many small SQLs and runs in every data nodes.The Final result is the aggregation of these all small SQLs. In this procedure, everything was computed in memory to get high performance.

In data warehouse layer, they use self-defined ETL tools to extract data from different databases to oracle-Exadata platform. Log-based data was put into hadoop and hbase.

This Log Buffer Edition rides on the wave of new ideas in the database realms. From Oracle technologies through MySQL to the SQL Server, things are piping hot.

Oracle:

The Call for Papers for the Rittman Mead BI Forum 2015 is currently open, with abstract submissions open to January 18th 2015.

The replicat process is the apply process within the Oracle GoldenGate environment.

“SELECT * FROM TABLE” Runs Out Of TEMP Space.

“log file sync” and the …

#DBHangOps 01/22/15 -- MySQL Event Scheduler, InnoDB IO Capacity, and more!

Hello everybody!

Join in #DBHangOps this Thursday, January, 22, 2015 at 11:00am pacific (18:00 GMT), to participate in the discussion about:

• MySQL Event Scheduler

  • What use cases do you have?
• What is everyone's understanding of innodb_io_capacity?

• Percona Live MySQL Conference and Expo 2015

  • What are you excited about?

You can check out the event page at https://plus.google.com/events/c50qple2vajgq6ltrutk1s3cfss on Thursday to participate.

As always, you can still watch the …

#DBHangOps 02/05/15 -- MySQL Event Scheduler, InnoDB IO Capacity, and more!

Hello everybody!

Join in #DBHangOps this Thursday, February, 5, 2015 at 11:00am pacific (18:00 GMT), to participate in the discussion about:

• Managing Multiple instance of MySQL

• MySQL Event Scheduler

  • What use cases do you have?
• What is everyone's understanding of innodb_io_capacity?

• Percona Live MySQL Conference and Expo 2015

  • What are you excited about?

You can check out the event page at https://plus.google.com/events/c50qple2vajgq6ltrutk1s3cfss on Thursday to participate.

As always, you can still watch the …

For some time already, FromDual's "Our Team" page lists me, and it even reveals that I joined in September, 2014. Also for some time, the list of FromDual blogs contains an entry "Jörg's Blog", but it doesn't lead to any entries. It is high time to fix this and create entries, starting with an introduction of myself.

Often, in such introductions people use the phrase of "the new kid on the block". I won't. If I am to use those words, I will arrange them as "the kid on the new block". The reason is that I don't feel as a new kid in the MySQL village (or is it a city?), let alone in DBMS country.

Ever since I left university (Technical University of Berlin, Germany), I have been involved in SQL DBMS development. After my previous product's team had been dissolved in Berlin and maintenance moved to Riga, Latvia, as a cost-cutting measure, I joined MySQL AB in 2004 as a member of the Build Team. Some of you will remember my name from …

A few days ago I wrote about MySQL performance implications of InnoDB isolation modes and I touched briefly upon the bizarre performance regression I found with InnoDB handling a large amount of versions for a single row. Today I wanted to look a bit deeper into the problem, which I also filed as a bug.

First I validated in which conditions the problem happens. It seems to happen only in REPEATABLE-READ isolation mode and only in case there is some hot rows which get many row versions during a benchmark run. For example the problem does NOT happen if I run sysbench with “uniform” distribution.

In terms of concurrent selects it also seems to require some very special conditions – you need to have the connection to let some …