October 27, 2014 By Severalnines

The term data warehousing often brings to mind things like large complex projects, big businesses, proprietary hardware and expensive software licenses. With Hadoop came open source data analysis software that ran on commodity hardware, this helped address at least some of the cost aspects. We had previously blogged about MongoDB and MySQL to Hadoop. But setting up and maintaining a Hadoop infrastructure might still be out of reach for small businesses or small projects with limited budgets. Well, perhaps then you might want to have a look at Redshift.

…

This week’s log buffer edition collects some of the insightful blog posts from Oracle, SQL Server, and MySQL.

Oracle:

Oracle StorageTek T10000D Tape Drive achieves FIPS 140-2 Validation.

Oracle is a Leader in IDC MarketScape for Global Trade Management.

The Benefits of Integrating a Google Search Appliance with an Oracle WebCenter or Liferay Portal.

Maintenance Windows is too small? Autotask Jobs fail.

SOA Suite 12c: Querying …

Mon, 2014-10-27 10:21hartmut

This plugin logs long running transactions to the server error log when transaction took more than a configurable number of seconds.

https://github.com/hholzgra/transaction_time_audit

Motivation

A customer was looking for a solution to track down long running transactions. After discussing some alternatives we decided that an Audit Plugin was the best approach towards this.

An Audit Plugin has the advantage that it:

• Is executed within the mysqld server. Its access to server internals via the official API is limited, but it turned out that the existing API provided all the building blocks we needed for a basic implementation:

   

  • the full stream of executed queries is visible
  • current transaction ID is visible …

Mon, 2014-10-27 10:21hartmut

This plugin logs long running transactions to the server error log when transaction took more than a configurable number of seconds.

https://github.com/hholzgra/transaction_time_audit

Motivation

A customer was looking for a solution to track down long running transactions. After discussing some alternatives we decided that an Audit Plugin was the best approach towards this.

An Audit Plugin has the advantage that it:

• Is executed within the mysqld server. Its access to server internals via the official API is limited, but it turned out that the existing API provided all the building blocks we needed for a basic implementation:

   

  • the full stream of executed queries is visible
  • current transaction ID is visible …

Let me start with a little story. You sit in your house near the fireplace in the living room and need a book from the library… Eh, no, sorry, wrong century. You’re building a robotic arm that will open your beer or brew your coffee or supply you with whatever other drinks of your choice… while you’ll be building the next robotic arm. So, you — soldering iron in one hand and Arduino in another — ask your little brother to bring a box with specific resistors (that you unexpectedly run out of) from the cellar. The problem — your brother is small and cannot tell a resistor from a respirator. You explain that it’s small thing with two wires sticking out of it. And he starts going back and forth brining you boxes after boxes of different small things with two wires.

This is approximately where we were in MySQL when NDB Cluster was just added. The use wants to find a row, say WHERE number_of_wires=2 AND size='small' AND …

In the previous blog posts I've talked about transactions which block other transactions but don't do anything and about some possible solutions.

In this post I will show you how to get even more information about what is locked by a transaction.

As you might have noticed the information_schema.innodb_locks table doesn't show all locks. This is what the documentation says:
"The INNODB_LOCKS table contains information about each lock that an InnoDB transaction has requested but not yet acquired, and each lock that a transaction holds that is blocking another …

OpenSQLCamp Boston has only been over for a week, but I already have about 2/3 of the videos uploaded to YouTube.  I have updated the schedule page with all the videos and slides I knew about.  I welcome comments with more information (e.g. links to slides, or tag or description suggestions for the YouTube videos).

Here’s the list of videos and slides so far (also linked at http://opensqlcamp.org/Events/Boston2010/Schedule):

Adventures in Alternative Energy “Data Monitoring” with MySQL — architecture and design case study – Matt Yonkovit, Percona – video

Cassandra and Lucene – Jake Luciani, Riptano –  …

I don’t think it’ll surprise anybody if I say that MariaDB or MySQL server knows a lot more about server system variables, then just their values. Indeed, every variable can be session or global only, read-only or writable, it has an associated help text (that is printed on mysqld --help --verbose), certain variables only accept values from a given set of strings (this set of allowed values is also printed in mysqld --help --verbose since MariaDB 10.1.0), numeric variables have lower and upper range boundaries of valid values (that are never printed anywhere), and so on. I always thought it’s kind of a waste that there is no way to query this information. That could’ve been very convenient, in particular for various GUI clients — they could show the help in tooltips, validate values and so on.

But recently we’ve got our users asking for it — precisely, for system variable metadata, whether a variable …

MariaDB 10.1.1 was recently released, and is available for download here:

https://downloads.mariadb.org/mariadb/10.1.1/

This is the second alpha release of MariaDB 10.1, so there are a lot of new changes and functionalities added, and many, many bugs fixed (I counted 637). Since it’s alpha, I’ll only cover the major changes and additions, as there are a lot of great new features, and omit covering any of the bug fixes (feel free to browse them all here).

To me, these are the highlights of the new features:

• InnoDB: You can now use OPTIMIZE TABLE to defragment InnoDB tablespaces (merged the Facebook/Kakao defragmentation patch). (Good blog post …

In my previous post I talked about a transaction which blocked other transactions without doing anything. I talked about finding data from the blocking transaction using SYS and performance_schema.

But what are the possible solutions?

The first solution is to (automatically) kill the blocking transactions. Long running transactions can also stall the purging in InnoDB. See this blog post by Mark Leith about a possible solution.

The second solution would be make the application end the transaction sooner and/or to commit more often. Depending on your application this might or might not work. I consider this the best solution.

The …