I published an SQLite3 storage back-end to NIST::NVD on the CPAN. It’s pretty quick. About as fast as the DB_File one, but without the down side of being tied to DB_File. It shouldn’t be too difficult to re-factor this code to any DBI-based database. MariaDB anyone?

I know it works on Debian. The nightly CPAN test results should come back shortly and I’ll find out how well it works on other platforms.

MySQL, like a lot of other software, has many knobs you can tweak. Most of these knobs may affect behaviour, but more importantly most affect the memory usage of the server, so getting these settings right is very important.

Most of MySQL’s memory is really used just as a cache, in one form or another, information that otherwise is on disk. So ensuring you have as large a cache as possible is important. However, making these memory sizes too large will trigger the server to start swapping and possibly can cause it to crash or cause the kernel to kill the process when it runs out of memory.  So that’s something we want to avoid.

Certain settings affect memory allocation on a per connection/thread basis, being bounded by thread_cache_size and max_connections.  If you configure for the worst behaviour (max_connections) you may end up not actually using all the memory you have available, memory which normally could be …

At the IOUC leaders’ summit in San Francisco this week, key leaders from Oracle, Java and MySQL user groups world wide have been meeting. This has included the key Oracle MySQL resources from the community, marketing and product teams. The Java User Groups and MySQL User Groups have been well represented and there has been very welcoming discussion with the IOUC about how we can become active within the Oracle Community.

There has been key discussions of upcoming and proposed MySQL events including the great outreach by the Oracle MySQL team with existing Open Source conferences this year including Scale, FOSDEM and South East Linuxfest just to name a few.

You can see the upcoming events at …

As a consultant I often use the MySQL Reference Manual to provide additional information for clients. I am very happy to recognize the quality of the content in the MySQL documentation, but why is the searching of the manual so, so bad?

While reading the General Security Issues section of the MySQL 5.5 manual, I performed a search for “CREATE USER”. I was not asking for anything abstract, this is an actual SQL command. I was rather horrified to find that the results could not even list the appropriate manual page in the first page of results.

I am not an expert in full-text search, however it does not take a rocket scientist to realize that a SQL keyword, the title of a page, in the language of the current page (English) and the current version of the Manual (5.5) should be …

You might have noticed that this blog changed name and URL. Instead of having one single blog for everything, I decided to split my blogging activities in two separate blogs.

If you're interested in MySQL, Oracle, Databases, Business Intelligence, Open Source, Cloud, etc. this is the place to be. Blog's new name is Data & Co.
A big "Thank you!" to David Stokes who moved all the planet.mysql.com references to the old blog to this new one.

On the other hand, if you'd like to read about communication, marketing, advertising, PR, soft skills, etc. feel free look at Publicime.  Please see the introduction of my new blog …

If you’ve been tuning your MySQL database and have wondered what effect the innodb_flush_method settings have on write performance, then this information might help. I’ve recently been doing a lot of baseline load tests to show performance differences between localdisk and the new SAN we’re deploying. Since we run InnoDB for everything in production, and writes are very heavy, I decided to run comparison tests between two identical servers to find the best setting for innodb_flush_method. We have the following specs for the hardware:

• Dell R610
• 24 core Intel Xeon X5670 @ 2.93ghz
• 72GB ECC RAM
• Brocade 825 HBA
• Local disk: RAID-10 15K SAS Ext3 (ugh)
• SAN: Oracle 7420 with four Intel Xeon X7550 @ 2.00GHz, 512GB RAM, 2TB read-cache(MLC-SSD), 36GB write cache (SLC-SSD), 3 disk shelves populated with 60x2TB 7200RM SATA drives setup in mirrored format with striped …

In my opinion, The Server Option and Variable Reference at http://dev.mysql.com/doc/refman/5.5/en/mysqld-option-tables.html rates as my most important page. This is a consolidated index that enables a drill down to the Server Command Options, System Variables, Startup and replication specifics, as well as important information on default values and differences between versions including point releases.

However, there is another page not in the actual manual, but at http://dev.mysql.com/doc/mysqld-version-reference/en/mysqld-version-reference-optvar.html which is an Options/Variables reference akin to the Reference Manual, but includes a 5.x version matrix.

Recently I was asked about some options that had to be …

I see so many posts on what happened to company X, product Y and dream Z that I couldn't resist the temptation to summarize this great year for MySQL. At the end of 2010, Oracle did an announcement we were all waiting for: MySQL 5.5 is GA! Another year has passed since then and it's time to reflect on what has been done.

I know this is a long post. I tried to rewrite it at least 10 times to make it shorter, but I couldn't condense the list. Hence, I wrote a summary in the beginning for those who don't want to read it all.

I believe that 2011 was an exceptional year for MySQL and I really enjoy being part of this team. I wish all of us a lot of success and fun in the years to come!

Summary:
Oracle released many  …

When using the replication slave stream, or mysql command line client and mysqlbinlog output from a binary/relay log, all statements are executed in a single thread as quickly as possible.

I am seeking a tool to simulate the replay of the binary/relay log for a benchmark at a pace that is more representative to original statements. For a simple example, if the Binary Log has 3 transactions in the first second, 2 transactions in the second second, and 5 transactions in the third second, I am wanting to simulate the replay to take roughly 3 seconds, not as fast as possible (which would be sub-second). The tool should try to wait the remainder of a second before processing SQL statements in the incoming stream.

Does anybody know of a tool that currently provides this type of functionality? Any input appreciated before I create my own.

For a work project, I wrote a library in perl that can be used to query the NVD feed that NIST publishes here:

http://nvd.nist.gov/download.cfm

Here’s a snippit from the perldoc:

use NIST::NVD::Query;
 
# use convert_nvdcve to generate these files from the XML dumps at
# http://nvd.nist.gov/download.cfm
 
my( $path_to_db, $path_to_idx_cpe ) = @ARGV;
 
my $q = NIST::NVD::Query->new( database => $path_to_db,
                               idx_cpe  => $path_to_idx_cpe,
                              );
 
# Given a Common Platform Enumeration urn, returns a list of known
# CVE IDs
 
my $cve_id_list = $q->cve_for_cpe( cpe => 'cpe:/a:zaal:tgt:1.0.6' );
 
my @entry;
 
foreach my $cve_id ( @$cve_id_list ){
 
  # Given a CVE ID, returns a CVE entry
 
  my $entry = $q->cve( cve_id => …