I don’t want to write a long long speech about my simple life…
I just want to clarify why I do what I do.

Many people have asked or wondered without asking why I do what I do
(Jeremy Cole - 2013)

A few years ago Ashley Unitt asked me what I was most proud of, and now, I can make a complete answer. I’m very proud to take part of a community, MySQL has transformed my job into a passion and an incredible desire to share this passion, as honestly as possible.

What does it mean in real life?

If I can help in decision making or take part of a discussion, without any bad motivations, independently, I do it!

If the opportunity arises to have a good time over a glass of rosé and discussing open source with Stéphane Varoqui, I do it!

If I have suddenly an idea and a friend with me for several nights and weekends to achieve it, oh yes, I do it!

I do it because it’s terribly exciting and fun, even if I’ve just did it for myself… on a whim.

Yes, I’m very proud when we made MYXPLAIN.net with Max, I’m proud to work for an amazing company, I’m proud to advise someone who challenge me from the other side of the world, I’m proud to read an email from Rick James in my inbox…

I’m proud to be honest!

You can take this 5-day live instructor-led course as a:

• Live-Virtual Event: Take this course from your own desk, no travel required. You can choose from a wide selection of events on the schedule to suit different timezones.
• In-Class Event: Travel to an education center to attend this class. Below is a selection of events already on the schedule.

Location	Date	Delivery Language
Brussels, Belgium	3 June 2013	English
London, England	9 September 2013	English
Aix-en-Provence, France	2 December 2013	French
Bordeaux, France	2 December 2013	French
Nice, France	4 November 2013	French
Puteaux, France	16 September 2013	French
Strasbourg, France	1 July 2013	French
Dresden, Germany	3 June 2013	German
Dusseldorf, Germany	24 June 2013	German
Gummersbach, Germany	1 July 2013	German
Hamburg, Germany	24 June 2013	German
Munchen, Germany	19 August 2013	German
Munster, Germany	9 September 2013	German
Stuttgart, Germany	8 July 2013	German
Budapest, Hungary	4 November 2013	Hungarian
Belfast, Ireland	24 June 2013	English
Milan, Italy	7 October 2013	Italian
Rome, Italy	17 June 2013	Italian
Utrecht, Netherlands	24 June 2013	Dutch
Warsaw, Poland	5 August 2013	Polish
Lisbon, Portugal	16 September 2013	European Portugese
Cairo, Egypt	30 June 2013	English
Nairobi, Kenya	22 July 2013	English
Jakarta, Indonesia	16 September 2013	English
Petaling Jaya, Malaysia	1 July 2013	English
Makati City, Philippines	3 June 2013	English
Melbourne, Australia	3 June 2013	English
Curitiba, Brazil	27 May 2013	Brazilian Portugese
Sao Paolo, Brazil	10 June 2013	Brazilian Portugese
Mexico City, Mexico	24 June 2013	Spanish

To register for this course or to learn more about the courses on the authentic MySQL curriculum, go to http://oracle.com/education/MySQL.

Based on my experiences solving tough SQL problems for Percona training and consulting, I’ll classify several common types of queries with which developers struggle. I’ll test several SQL solutions for each type of query objective, and show how you can use MySQL 5.6 built-in methods to analyze them for optimal query efficiency.  The discussion will cover optimizer reports, query profiling, and session status to measure performance.

The query patterns will include:

• Exclusion Join
• Random Selection
• Greatest-Per-Group
• Dynamic Pivot
• Relational Division

Please register for this webinar and join me next Friday!

The post Webinar: SQL Query Patterns, Optimized appeared first on MySQL Performance Blog.

However here around eight months later, it is starting to shape up. While there definitely still is work to be done, e.g. creating the full tree and outputting it in text mode (more on modes later) takes around one minute on my test system – granted I am using a standard laptop and MySQL is running in a VM, so it is nothing sophisticated.

The current routines can be found in ps_tools.sql.gz – it may later be merged into Mark Leith’s ps_helper to try to keep the Performance Schema tools collected in one place.

Note: This far the routines have only been tested in Linux on MySQL 5.6.11. Particularly line endings may give problems on Windows and Mac.

Description of the ps_tools Routines

The current status are two views, four stored procedure, and four functions – not including several functions and procedures that does all the hard work:

• Views:
  • setup_consumers – Displays whether each consumer is enabled and whether the consumer actually will be collected based on the hierarchy rules described in Pre-Filtering by Consumer in the Reference Manual.
  • accounts_enabled – Displays whether each account defined in the mysql.user table has instrumentation enabled based on the rows in performance_schema.setup_actors.
• Procedures:
  • setup_tree_consumers(format, color) – Create a tree based on setup_consumers displaying whether each consumer is effectively enabled. The arguments are:
    • format is the output format and can be either (see also below).:
      • Text: Left-Right
      • Text: Top-Bottom
      • Dot: Left-Right
      • Dot: Top-Bottom
    • color is whether to add bash color escapes sequences around the consumer names when used a Text format (ignored for Dot outputs).
  • setup_tree_instruments(format, color, only_enabled, regex_filter) – Create a tree based on setup_instruments displaying whether each instrument is enabled. The tree is creating by splitting the instrument names at each /. The arguments are:
    • format is the output format and can be either:
      • Text: Left-Right
      • Dot: Left-Right
      • Dot: Top-Bottom
    • color is whether to add bash color escapes sequences around the instrument names when used a Text format (ignored for Dot outputs).
    • type – whether to base the tree on the ENABLED or TIMED column of setup_instruments.
    • only_enabled – if TRUE only the enabled instruments are included.
    • regex_filter – if set to a non-empty string only instruments that match the regex will be included.
  • setup_tree_actors_by_host(format, color) – Create a tree of each account defined in mysql.user and whether they are enabled; grouped by host. The arguments are:
    • format is the output format and can be either:
      • Text: Left-Right
      • Dot: Left-Right
      • Dot: Top-Bottom
    • color is whether to add bash color escape sequences around the names when used a Text format (ignored for Dot outputs).
  • setup_tree_actors_by_user – Create a tree of each account defined in mysql.user and whether they are enabled; grouped by username. The arguments are:
    • format is the output format and can be either:
      • Text: Left-Right
      • Dot: Left-Right
      • Dot: Top-Bottom
    • color is whether to add bash color escape sequences around the names when used a Text format (ignored for Dot outputs).
• Functions:
  • is_consumer_enabled(consumer_name) – Returns whether a given consumer is effectively enabled.
  • is_account_enabled(host, user) – Returns whether a given account (host, user) is enabled according to setup_actors.
  • substr_count(haystack, needle, offset, length) – The number of times a given substring occurs in a string. A port of the PHP function of the same name.
  • substr_by_delim(set, delimiter, pos) – Returns the Nth element from a delimiter string.

The two functions substr_count() and substr_by_delim() was also described in an earlier blog.

The formats for the four stored procedures consists of two parts: whether it is Text or Dot and the direction. Text is a tree that can be viewed directly either in the mysql command line client (coloured output not supported) or the shell (colours supported for bash). Dot will output a DOT graph file in the same way as dump_thread_stack() in ps_helper. The direction is as defined in the DOT language, so e.g. Left-Right will have the first level furthest to the left, then add each new level to the right of the parent level.

Examples

As the source code – including comments – is more than 1600 lines, I will not discuss it here, but rather go through some examples.

setup_tree_consumers

Using the coloured output:

or the same using a non-coloured output:

setup_tree_instruments

Here a small part of the tree is selected using a regex.

setup_tree_actors_%

With only root@localhost and root@127.0.0.1 enabled, the outputs of setup_tree_actors_by_host and setup_tree_actors_by_user gives respectively:

DOT Graph of setup_instruments

The full tree of setup_instruments can be created using the following sequence of steps (I am using graphviz to get support for dot files):

MySQL 5.6.11$ echo -e "$(mysql -NBe "CALL ps_tools.setup_tree_instruments('Dot: Left-Right', FALSE, 'Enabled', FALSE, '')")" > /tmp/setup_instruments.dot
MySQL 5.6.11$ dot -Tpng /tmp/setup_instruments.dot -o /tmp/setup_instruments.png

The full output is rather large (6.7M). If you want to see if you can get to it at http://mysql.wisborg.dk/wp-content/uploads/2013/05/setup_tree_instruments_dot_lr.png.

Views

mysql> SELECT * FROM ps_tools.setup_consumers;
+--------------------------------+---------+----------+
| NAME                           | ENABLED | COLLECTS |
+--------------------------------+---------+----------+
| events_stages_current          | NO      | NO       |
| events_stages_history          | NO      | NO       |
| events_stages_history_long     | NO      | NO       |
| events_statements_current      | YES     | YES      |
| events_statements_history      | NO      | NO       |
| events_statements_history_long | NO      | NO       |
| events_waits_current           | NO      | NO       |
| events_waits_history           | NO      | NO       |
| events_waits_history_long      | NO      | NO       |
| global_instrumentation         | YES     | YES      |
| thread_instrumentation         | YES     | YES      |
| statements_digest              | YES     | YES      |
+--------------------------------+---------+----------+
12 rows in set (0.00 sec)

mysql> SELECT * FROM ps_tools.accounts_enabled;
+-------------+-----------+---------+
| User        | Host      | Enabled |
+-------------+-----------+---------+
| replication | 127.0.0.1 | NO      |
| root        | 127.0.0.1 | YES     |
| root        | ::1       | NO      |
| meb         | localhost | NO      |
| memagent    | localhost | NO      |
| root        | localhost | YES     |
+-------------+-----------+---------+
6 rows in set (0.00 sec)

Conclusion

There is definitely more work to do on making the Performance Schema easier to access. ps_helper and ps_tools are a great start to what I am sure will be an extensive library of useful diagnostic queries and tools.

Duplicate Indexes

“The server now issues a warning if an index is created that duplicates an existing index, or an error in strict SQL mode.” Bug #37520

Example Testcase:

mysql> SHOW CREATE TABLE city\G
*************************** 1. row ***************************
       Table: city
Create Table: CREATE TABLE `city` (
  `ID` int(11) NOT NULL AUTO_INCREMENT,
  `Name` char(35) NOT NULL DEFAULT '',
  `CountryCode` char(3) NOT NULL DEFAULT '',
  `District` char(20) NOT NULL DEFAULT '',
  `Population` int(11) NOT NULL DEFAULT '0',
  PRIMARY KEY (`ID`),
  KEY `CountryCode` (`CountryCode`),
  CONSTRAINT `city_ibfk_1` FOREIGN KEY (`CountryCode`) REFERENCES `Country` (`Code`)
) ENGINE=InnoDB AUTO_INCREMENT=4080 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

mysql> ALTER TABLE city add index (countrycode);
ERROR 1831 (HY000): Duplicate index 'CountryCode_2' defined on the table 'world.city'. 
This is deprecated and will be disallowed in a future release.

Pretty cool - I know this previously caught a lot of people.

Control-C support in the client

“Previously, Control+C in mysql interrupted the current statement if there was one, or exited mysql if not. Now Control+C interrupts the current statement if there was one, or cancels any partial input line otherwise, but does not exit.” Bug #66583

Example Testcase:

mysql> this is a test
    -> test
    -> test
    -> ^C

So if I want to quit, I can now control-C then type “quit”. This is much more intuitive.

• Complete functional and non functional test coverage of changed and new functionality
• No regressions
• More than 80% Code Coverage

• MTR(mysql-test-run)- Used to cover simple to medium complexity test cases
• RQG(Random Query Generator) - This is a very powerful tool and is used for concurrency tests, generating large number of SQL queries etc

• No open bugs in the new feature - This is very strictly enforced and even minor bugs are not allowed. We believe that bugs are easiest to fix when the code is new and hence this can help us deliver features that are of high quality.
• No regressions - A feature gets developed on a tree which gets tested regularly through a continuous integration testing tool. Any regressions are detected and fixed before sign off.
• Acceptable Code coverage numbers - Code coverage report is generated for the changed lines of code and the minimum expected coverage is 80%. Most features have coverage of more than 90%. Any uncovered lines of code are analyzed and wherever possible new tests are added to increase code coverage.
• All new tests are added to the automated regression suite.
• All planned testing has completed with satisfactory results.

This release fixes number of high-priority bugs since version 2.1 became GA. It’s advised to upgrade your latest 2.1 version to 2.1.2. This release is the latest stable release in the 2.1 series.

Bugs Fixed:

• Using Perl’s DBD::MySQL package for server communication instead of spawning the MySQL command line client introduced a regression which caused innobackupex –galera-info option to fail. Bug fixed #1180672.
• The format of xtrabackup_galera_info was missing the ‘:’ separator between the values of wsrep_local_state_uuid and wsrep_last_committed. Bug fixed #1181222.
• innobackupex automatic version detection did not work correctly for latest Percona Server and MySQL 5.1 releases which could cause innobackupex to fail. Bugs fixed #1181092, #1181099 and #1180905.
• When backing up a server that is not a replication slave with the innobackupex –slave-info option, innobackupex failed with a fatal error. Replaced the fatal error with a diagnostic message about innobackupex –slave-info being ignored in such a case. Bug fixed #1180662.
• Low values for wait_timeout on the server could cause server to close the connection while backup is being taken. Fixed by setting the bigger value for wait_timeout option on the server to prevent server from closing connections if the global wait_timeout value is set too low. Bug fixed #1180922.

Other bug fixes: bug fixed #1177182.

Release notes with all the bugfixes for Percona XtraBackup 2.1.2 are available in our online documentation. Bugs can be reported on the launchpad bug tracker.

The post Percona XtraBackup 2.1.2 for MySQL available for download appeared first on MySQL Performance Blog.

Events
DB Hangops - every other Wednesay at noon Pacific time
Upcoming MySQL events

Training
SkySQL Trainings
Tungsten University trainings

read more

The red carpet has been laid down at this Log Buffer Edition, and you can witness and cheer the cat-walking blog posts from Oracle, SQL Server and MySQL. Every one of them is chic, elegant, sensual in its own right. Enjoy.
Oracle:

Create colored heat maps in SQL*Plus with Kyle Hailey.

Here’s a quick and dirty script to create a procedure (in the SYS schema – so be careful) to check the Hakan Factor for an object.

Connor has a good post about default null for collection parameter.

This is yet another blogpost on Oracle’s direct path read feature which was introduced for non-parallel query processes in Oracle version 11.

Owen Allen has seen some questions about provisioning Oracle Solaris 11. They boil down to this.

SQL Server:

Shashank Srivastava tells us as how to Change the SQL Server Instance Name after Renaming the Windows Host.

Daniel Calbimonte shares as how to synchronize two SSAS Servers.

Data Architecture underpins just about everything we do in IT.  Without a clear understanding of how data is structured, there is no reliable way to derive meaning from it.

Orlando Colamatteo is login-less in Seattle.

Lets get started testing database with tSQLt with Robert Sheldon.

MySQL:

After a lot of fuzz, Anders Karlsson is now releasing MyQuery version 3.5.1.

Nothing like reestablishing a tradition and Dave Stokes is doing just that for MySQL.

Mare Alff is spreading the word about the performance schema.

Slava Akhmechet talks about secondary indexes, batched inserts performance improvements, soft durability mode.

It is a central part of the MySQL philosophy to try and help you as much as you can. There are many occasions when it could tell you that what you are asking for is utterly stupid or give you a bad execution plan because “you asked for it”.

Disclosure: The research and testing conducted for this post were sponsored by Virident.

First, some background information. All tests were conducted on Percona’s Cisco UCS C250 test machine, and both the vCache and FlashCache tests used the same 2.2TB Virident FlashMAX II as the cache storage device. EXT4 is the filesystem, and CentOS 6.4 the operating system, although the pre-release modules I received from Virident required the use of the CentOS 6.2 kernel, 2.6.32-220, so that was the kernel in use for all of the benchmarks on both systems. The benchmark tool used was sysbench 0.5 and the version of MySQL used was Percona Server 5.5.30-rel30.1-465. Each test was allowed to run for 7200 seconds, and the first 3600 seconds were discarded as warmup time; the remaining 3600 seconds were averaged into 10-second intervals. All tests were conducted with approximately 78GiB of data (32 tables, 10M rows each) and a 4GiB buffer pool. The cache devices were flushed to disk immediately prior to and immediately following each test run.

With that out of the way, let’s look at some numbers.

vCache vs. vCache – MySQL parameter testing

The first test was designed to look solely at vCache performance under some different sets of MySQL configuration parameters. For example, given that the front-end device is a very fast PCIe SSD, would it make more sense to configure MySQL as if it were using SSD storage or to just use an optimized HDD storage configuration? After creating a vCache device with the default configuration, I started with a baseline HDD configuration for MySQL (configuration A, listed at the bottom of this post) and then tried three additional sets of experiments. First, the baseline configuration plus:

innodb_read_io_threads = 16
innodb_write_io_threads = 16

We call this configuration B. The next one contained four SSD-specific optimizations based partially on some earlier work that I’d done with this Virident card (configuration C):

innodb_io_capacity = 30000
innodb_adaptive_flushing_method = keep_average
innodb_flush_neighbor_pages=none
innodb_max_dirty_pages_pct = 60

And then finally, a fourth test (configuration D) which combined the parameter changes from tests B and C. The graph below shows the sysbench throughput (tps) for these four configurations:

As we can see, all of the configuration options produce numbers that, in the absence of outliers, are roughly identical, but it’s configuration C (shown in the graph as the blue line – SSD config) which shows the most consistent performance. The others all have assorted performance drops scattered throughout the graph. We see the exact same pattern when looking at transaction latency; the baseline numbers are roughly identical for all four configurations, but configuration C avoids the spikes and produces a very constant and predictable result.

vCache vs. FlashCache – the basics

Once I’d determined that configuration C appeared to produce the most optimal results, I moved on to reviewing FlashCache performance versus that of vCache, and I also included a “no cache” test run as well using the base HDD MySQL configuration for purposes of comparison. Given the apparent differences in time-based flushing in vCache and FlashCache, both cache devices were set up so that time-based flushing was disabled. Also, both devices were set up such that all IO would be cached (i.e., no special treatment of sequential writes) and with a 50% dirty page threshold. Again, for comparison purposes, I also include the numbers from the vCache test where the time-based flushing is enabled.

As we’d expect, the HDD-only solution barely registered on the graph. With a buffer pool that’s much smaller than the working set, the no-cache approach is fairly crippled and ineffectual. FlashCache does substantially better, coming in at an average of around 600 tps, but vCache is about 3x better. The interesting item here is that vCache with time-based flushing enabled actually produces better and more consistent performance than vCache without time-based flushing, but even at its worst, the vCache test without time-based flushing still outperforms FlashCache by over 2x, on average.

Looking just at sysbench reads, vCache with time-based flushing consistently hit about 27000 per second, whereas without time-based flushing it averaged about 12500. FlashCache came in around 7500 or so. Sysbench writes came in just under 8000 for vCache + time-based flushing, around 6000 for vCache without time-based flushing, and somewhere around 2500 for FlashCache.

We can take a look at some vmstat data to see what’s actually happening on the system during all these various tests. Clockwise from the top left in the next graph, we have “no cache”, “FlashCache”, “vCache with no time-based flushing”, and “vCache with time-based flushing.” As the images demonstrate, the no-cache system is being crushed by IO wait. FlashCache and vCache both show improvements, but it’s not until we get to vCache with the time-based flushing that we see some nice, predictable, constant performance.

So why is it the case that vCache with time-based flushing appears to outperform all the rest? My hypothesis here is that time-based flushing allows the backing store to be written to at a more constant and, potentially, submaximal, rate compared to dirty-page-threshold flushing, which kicks in at a given level and then attempts to flush as quickly as possible to bring the dirty pages back within acceptable bounds. This is, however, only a hypothesis.

vCache vs. FlashCache – dirty page threshold

Finally, we examine the impact of a couple of different dirty-page ratios on device performance, since this is the only parameter which can be reliably varied between the two in the same way. The following graph shows sysbench OLTP performance for FlashCache vs. vCache with a 10% dirty threshold versus the same metrics at a 50% dirty threshold. Time-based flushing has been disabled. In this case, both systems produce better performance when the dirty-page threshold is set to 50%, but once again, vCache at 10% outperforms FlashCache at 10%.

The one interesting item here is that vCache actually appears to get *better* over time; I’m not entirely sure why that’s the case or at what point the performance is going to level off since these tests were all run for 2 hours anyway, but I think the overall results still speak for themselves, and even with a vCache volume where the dirty ratio is only 10%, such as might be the case where a deployment has a massive data set size in relation to both the working set and the cache device size, the numbers are encouraging.

Conclusion

Overall, the I think the graphs speak for themselves. When the working set outstrips the available buffer pool memory but still fits into the cache device, vCache shines. Compared to a deployment with no SSD cache whatsoever, FlashCache still does quite well, massively outperforming the HDD-only setup, but it doesn’t even really come close to the numbers obtained with vCache. There may be ways to adjust the FlashCache configuration to produce better or more consistent results, or results that are more inline with the numbers put up by vCache, but when we consider that overall usability was one of the evaluation points and combine that with the fact that the best vCache performance results were obtained with the default vCache configuration, I think vCache can be declared the clear winner.

Base MySQL & Benchmark Configuration

All benchmarks were conducted with the following:

sysbench ­­--num­-threads=32 ­­--test=tests/db/oltp.lua ­­--oltp_tables_count=32 \
--oltp­-table­-size=10000000 ­­--rand­-init=on ­­--report­-interval=1 ­­--rand­-type=pareto \
--forced­-shutdown=1 ­­--max­-time=7200 ­­--max­-requests=0 ­­--percentile=95 ­­\
--mysql­-user=root --mysql­-socket=/tmp/mysql.sock ­­--mysql­-table­-engine=innodb ­­\
--oltp­-read­-only=off run


The base MySQL configuration (configuration A) appears below:

#####fixed innodb options
innodb_file_format = barracuda
innodb_buffer_pool_size = 4G
innodb_file_per_table = true
innodb_data_file_path = ibdata1:100M
innodb_flush_method = O_DIRECT
innodb_log_buffer_size = 128M
innodb_flush_log_at_trx_commit = 1
innodb_log_file_size = 1G
innodb_log_files_in_group = 2
innodb_purge_threads = 1
innodb_fast_shutdown = 1
#not innodb options (fixed)
back_log = 50
wait_timeout = 120
max_connections = 5000
max_prepared_stmt_count=500000
max_connect_errors = 10
table_open_cache = 10240
max_allowed_packet = 16M
binlog_cache_size = 16M
max_heap_table_size = 64M
sort_buffer_size = 4M
join_buffer_size = 4M
thread_cache_size = 1000
query_cache_size = 0
query_cache_type = 0
ft_min_word_len = 4
thread_stack = 192K
tmp_table_size = 64M
server­id = 101
key_buffer_size = 8M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
bulk_insert_buffer_size = 8M
myisam_sort_buffer_size = 8M
myisam_max_sort_file_size = 10G
myisam_repair_threads = 1
myisam_recover


The post Virident vCache vs. FlashCache: Part 2 appeared first on MySQL Performance Blog.

Thanks to Tim Callaghan for speaking Tuesday night at the Effective MySQL New York meetup on Fractal Tree Indexes : Theory and Practice (MySQL and MongoDB). There was a good turnout and a full room to learn how the TokuDB storage engine from Tokutek is changing how to handle big data in MySQL.

Also interesting is how the same technology has been applied for use in MongoDB including giving MongoDB transactions; a big change for NoSQL.

Related News: Tokutek Meets Big Data Demand With Open Source TokuDB

(This is part one of a two part series)

Disclosure: The research and testing for this post series was sponsored by Virident.

Usability is, to some extent, a subjective call, as I may have preferences for or against a certain mode of operation that others may not share, so readers may have a different opinion than mine, but on this point I call it an overall draw between vCache and FlashCache.

Ease of basic installation. The setup process was simply a matter of installing two RPMs and running a couple of commands to enable vCache on the PCIe flash card (a Virident FlashMAX II) and set up the cache device with the command-line utilities supplied with one of the RPMs. Moreover, the vCache software is built in to the Virident driver, so there is no additional module to install. FlashCache, on the other hand, requires building a separate kernel module in addition to whatever flash memory driver you’ve already had to install, and then further configuration requires modification to assorted sysctls. I would also argue that the vCache documentation is superior. Winner: vCache.

Ease of post-setup modification / advanced installation. Many of the FlashCache device parameters can be easily modified by echoing the desired value to the appropriate sysctl setting; with vCache, there is a command-line binary which can modify many of the same parameters, but doing so requires a cache flush, detach, and reattach. Winner: FlashCache.

Operational Flexibility: Both solutions share many features here; both of them allow whitelisting and blacklisting of PIDs or simply running in a “cache everything” mode. Both of them have support for not caching sequential IO, adjusting the dirty page threshold, flushing the cache on demand, or having a time-based cache flushing mechanism, but some of these features operate differently with vCache than with FlashCache. For example, when doing a manual cache flush with vCache, this is a blocking operation. With FlashCache, echoing “1″ to the do_sync sysctl of the cache device triggers a cache flush, but it happens in the background, and while countdown messages are written to syslog as the operation proceeds, the device never reports that it’s actually finished. I think both kinds of flushing are useful in different situations, and I’d like to see a non-blocking background flush in vCache, but if I had to choose one or the other, I’ll take blocking and modal over fire-and-forget any day. FlashCache does have the nice ability to switch between FIFO and LRU for its flushing algorithm; vCache does not. This is something that could prove useful in certain situations. Winner: FlashCache.

Operational Monitoring: Both solutions offer plenty of statistics; the main difference is that FlashCache stats can be pulled from /proc but vCache stats have to be retrieved by running the vgc-vcache-monitor command. Personally, I prefer “cat /proc/something” but I’m not sure that’s sufficient to award this category to FlashCache. Winner: None.

Time-based Flushing: This wouldn’t seem like it should be a separate category, but because the behavior seems to be so different between the two cache solutions, I’m listing it here. The vCache manual indicates that “flush period” specifies the time after which dirty blocks will be written to the backing store, whereas FlashCache has a setting called “fallow_delay”, defined in the documentation as the time period before “idle” dirty blocks are cleaned from the cache device. It is not entirely clear whether or not these mechanisms operate in the same fashion, but based on the documentation, it appears that they do not. I find the vCache implementation more useful than the one present in FlashCache. Winner: vCache.

Although nobody likes a tie, if you add up the scores, usability is a 2-2-1 draw between vCache and FlashCache. There are things that I really liked better with FlashCache, and there are other things that I thought vCache did a much better job with. If I absolutely must pick a winner in terms of usability, then I’d give a slight edge to FlashCache due to configuration flexibility, but if the GA release of vCache added some of FlashCache’s additional configuration options and exposed statistics via /proc, I’d vote in the other direction.

Stay tuned for part two of this series, wherein we’ll take a look at some benchmarks. There’s no razor-thin margin of victory for either side here: vCache outperforms FlashCache by a landslide.

The post Virident vCache vs. FlashCache: Part 1 appeared first on MySQL Performance Blog.

This release includes the long-awaited support for secondary indexes, a new algorithm for batched inserts that results in an ~18x performance improvement, support for soft durability (don't worry -- off by default), and over 180 bug fixes, features, and enhancements.

Secondary indexes

Support for secondary indexes has been the most requested feature since we launched RethinkDB, and has been in development for over six months. It required a massive amount of server work and involved modifying almost every part of the codebase: the BTree code, the concurrency subsystem, the distribution layer, the query language, the client drivers, and even the web UI.

We worked hard to make sure secondary indexes are extremely easy to use. Here's how you'd create a secondary index on the last_name attribute:

r.table('users').indexCreate('last_name')

Then getting all users with the last name Smith would be:

r.table('users').getAll('Smith', { index: 'last_name' })

Or you could retrieve arbitrary ranges of the index. For example, all users whose last names are between Smith and Wade:

r.table('users').between('Smith', 'Wade', { index: 'last_name' })

Listing and dropping indexes is also a part of the API:

r.table('users').indexList()             // list indexes on table 'users'
r.table('users').indexDrop('last_name')  // drop index 'last_name' on table 'users'

In addition to manipulating secondary indexes via ReQL, you can perform these from the admin UI:

You can define compound indexes, and even indexes based on arbitrary ReQL expressions. Secondary indexes can be used to do efficient table joins, and are sharded and replicated along with the rest of the database. Learn more about how to use secondary indexes in the FAQ.

Batched inserts performance improvements

Since the initial release, RethinkDB has supported inserting batches of documents. Instead of inserting multiple documents one at a time:

r.table('users').insert({ name: 'Michael' })
r.table('users').insert({ name: 'Bill' })

it was always possible to insert a batch of documents in one command with a single network roundtrip:

r.table('users').insert([{ name: 'Michael' },
                         { name: 'Bill' }])

However, the server had always executed batched inserts by translating them to a series of single insert commands. While sending the data in a single network roundtrip reduced the network latency, it still had very poor performance because the server would have to flush each document to disk before moving on to the next document.

The 1.5 release includes a new insert algorithm that flushes changes to disk in batches while maintaining the guarantee of consistency in case of power failure. This algorithm drastically improves performance of batched inserts. While not something we'd call a benchmark, inserting 100 medium-sized documents went from 2.8 seconds to 160 milliseconds on a development system, and the ~18x performance improvement is consistently reproducible on different batched insert workloads and types of hardware.

Soft durability mode

Early in the development of RethinkDB, we made the design decision to be extremely conservative about durability and safety of users's data. In this respect RethinkDB is like any other traditional database system -- the server does not acknowledge the write until it's safely committed to disk. While this is a sensible default for a database system, many of our users pointed out that they're using RethinkDB for storing access logs or clickstream data, or as a persistent, replicated cache of JSON documents. These scenarios might require trading off some durability guarantees for higher performance.

The 1.5 release includes support for relaxed durability (off by default, of course). For tables in this mode, the server acknowledges writes as soon as it receives them, and flushes data to disk in the background. Flushes normally occur every few seconds, or if there are too many dirty blocks cached in memory.

Hard durability can be turned off when creating a table via the advanced settings in the web UI or in ReQL:

db.tableCreate('http_logs', { hard_durability: false })

You'll notice that write performance on tables with hard durability turned off is about ~30x faster than normal tables. Note that the data still gets flushed to disk in the background and is consistent in case of failures. For those familiar with MySQL's InnoDB engine, RethinkDB's soft durability is similar to setting InnoDB's innodb_flush_log_at_trx_commit to 0 (more info on this setting in InnoDB).

You can change the durability mode for an existing table via the admin CLI:

$ rethinkdb admin -j localhost:29015
localhost:29015> set durability http_logs --hard

Many more enhancements

The 1.5 release includes many more enhancements -- check out the changelog for a complete list. Here are a few more notable changes:

• For super-fast insert performance, RethinkDB now supports noreply writes that allow writing data over the network without waiting for a server acknowledgment. This mode allows for an apples-to-apples benchmark comparison with MongoDB.
• The Data Explorer in the web UI now supports electric punctuation, as well as a toggle to turn it off. This can make typing queries in the data explorer much more pleasant.
• The web UI will now check if there is a new version of RethinkDB and inform you if you need to upgrade (the check is done as a simple AJAX request from the browser, and can be turned off).

Looking forward to 1.6

We are already hard at work on the 1.6 release. This release should be quick and easy, and will include many ReQL enhancements. We will also focus on continuously improving performance with each version. However, the next release isn't set in stone. If a feature is important to you, let us know.

MySQL has two great features which historical haven't played well together:

1. Index-only queries:  In some cases, MySQL can resolve a query directly from the index, without having to read the underlying table.
2. Prefix indexes:  This allows you to specify how many bytes to index, which can reduce index size or allow you to index the larger data types (ie. BLOB/TEXT).  The drawback being that the entire field isn't stored in the index, so you can't do index-only queries.

One common optimization we do to reduce IOP consumption on database servers is to add additional columns to indexes in order to allow more queries to be index-only.  However, sometimes we have these large TEXT fields in order to allow for larger content -- even if the content is normally very small.

For example:

CREATE TABLE tbl (

  id  INT PRIMARY KEY,

  other_column INT,

  txt_field MEDIUMTEXT,

  INDEX (other_column, txt_field(100))

) ENGINE=INNODB;

SELECT txt_field FROM tbl WHERE idx = 5;

MySQL will see that this is a prefix index and always have to read from the underlying table.  However in this case, if the content of txt_field is less than 100 bytes, we shouldn't have to actually read the underlying table since the data is still fully contained for that row! 

Our engineering team recently fixed this and it is available as a patch at:

https://github.com/facebook/mysql-5.6/commit/4dd11b503085548f1c00bc927c7ae8b5bcb21626

To enable this optimization, you can use:

SET global innodb_prefix_index_cluster_optimization=1;

And then we get a count of how many potential IOPS were saved due to this:

SHOW GLOBAL STATUS LIKE 'innodb_secondary_index_triggered_cluster_reads_avoided';

The best part is that this will work even when we have mixed situations, where some values are larger than the prefix and some are smaller.  Basically this is just always a good optimization (we can only disable it due to initial rollout testing).

I hope that other distributions of InnoDB/XtraDB pick up this optimization as well.

The content comes from my personal experiences over many years benchmarking and testing databases, usually focusing on performance.  It was an opportunity to see how far my personal benchmark infrastructure has evolved, but even better has inspired me to improve it in several areas.

I never had a chance to to my own post-conference wrap-up regarding the Percona Live show.  While waiting for my flight home at SFO airport I concluded that it was by far the best technology conference I’ve ever attended.  The MySQL community is truly amazing and finally meeting many of the people that I’ve only “known” online was great.  I’m really looking forward to next year’s event and ways I can contribute to existing benchmarking and testing projects.

mysql> SELECT CONCAT(T.TABLE_SCHEMA,'.',T.TABLE_NAME) AS TABLE_NAME,
    -> P.PARTITION_NAME AS PART,IBT.SPACE,IBD.PATH,T.DATA_FREE AS T_DATA_FREE,
    -> P.DATA_FREE AS P_DATA_FREE FROM INFORMATION_SCHEMA.TABLES T 
    -> LEFT JOIN INFORMATION_SCHEMA.PARTITIONS P ON P.TABLE_SCHEMA=T.TABLE_SCHEMA 
    -> AND P.TABLE_NAME=T.TABLE_NAME 
    -> LEFT JOIN INFORMATION_SCHEMA.INNODB_SYS_TABLES IBT 
    -> ON IBT.NAME=CONCAT(T.TABLE_SCHEMA,'/',T.TABLE_NAME) 
    -> OR IBT.NAME=CONCAT(T.TABLE_SCHEMA,'/',T.TABLE_NAME,'#P#',P.PARTITION_NAME) 
    -> LEFT JOIN INFORMATION_SCHEMA.INNODB_SYS_DATAFILES IBD 
    -> ON IBD.SPACE=IBT.SPACE WHERE ENGINE='InnoDB' ORDER BY T.TABLE_SCHEMA,T.TABLE_NAME;
+----------------------------+------+-------+-----------------------------------+-------------+-------------+
| TABLE_NAME                 | PART | SPACE | PATH                              | T_DATA_FREE | P_DATA_FREE |
+----------------------------+------+-------+-----------------------------------+-------------+-------------+
| innodbfreespacetest.t1     | NULL |     6 | ./innodbfreespacetest/t1.ibd      |     4194304 |     4194304 |
| innodbfreespacetest.t11    | p2   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t11    | p1   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t11    | p0   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t11    | p4   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t11    | p3   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t13    | NULL |    46 | ./innodbfreespacetest/t13.ibd     |     4194304 |     4194304 |
| innodbfreespacetest.t2     | NULL |     0 | NULL                              |    80740352 |    80740352 |
| innodbfreespacetest.t5     | p1   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t5     | p0   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t5     | p4   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t5     | p3   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t5     | p2   |     0 | NULL                              |   403701760 |    80740352 |
| innodbfreespacetest.t6     | p4   |    15 | ./innodbfreespacetest/t6#P#p4.ibd |           0 |           0 |
| innodbfreespacetest.t6     | p3   |    14 | ./innodbfreespacetest/t6#P#p3.ibd |           0 |           0 |
| innodbfreespacetest.t6     | p2   |    13 | ./innodbfreespacetest/t6#P#p2.ibd |           0 |           0 |
| innodbfreespacetest.t6     | p1   |    12 | ./innodbfreespacetest/t6#P#p1.ibd |           0 |           0 |
| innodbfreespacetest.t6     | p0   |    11 | ./innodbfreespacetest/t6#P#p0.ibd |           0 |           0 |
| mysql.innodb_index_stats   | NULL |     2 | ./mysql/innodb_index_stats.ibd    |           0 |           0 |
| mysql.innodb_table_stats   | NULL |     1 | ./mysql/innodb_table_stats.ibd    |           0 |           0 |
| mysql.slave_master_info    | NULL |     4 | ./mysql/slave_master_info.ibd     |           0 |           0 |
| mysql.slave_relay_log_info | NULL |     3 | ./mysql/slave_relay_log_info.ibd  |           0 |           0 |
| mysql.slave_worker_info    | NULL |     5 | ./mysql/slave_worker_info.ibd     |           0 |           0 |
+----------------------------+------+-------+-----------------------------------+-------------+-------------+
23 rows in set (0.05 sec)

Percona XtraBackup enables backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, XtraBackup drives down backup costs while providing unique features for MySQL backup. The new 2.1.1 GA version offers improved performance, enterprise-grade security, and lower resource usage.

This release is the first GA (Generally Available) stable release in the 2.1 series.

New Features:

• Percona XtraBackup now has support for Compact Backups. This feature can be used for taking the backups that will take less amount of disk space. GA release now contains new innobackupex –rebuild-threads option that can be used to specify the number of threads started by XtraBackup when rebuilding secondary indexes on innobackupex --apply-log --rebuild-indexes. This allows parallel processing of individual tables when rebuilding the index.
• Percona XtraBackup has implemented Encrypted Backups. This feature can be used to encrypt/decrypt both local and streamed backups in order to add another layer of protection to the backups.
• innobackupex now uses Perl’s DBD::MySQL package for server communication instead of spawning the MySQL command line client.
• Support for InnoDB 5.0 and InnoDB 5.1 builtin has been removed from Percona XtraBackup.
• After being deprecated in previous version, option --remote-host has been completely removed in Percona XtraBackup 2.1.
• Percona XtraBackup can use XtraDB changed page tracking feature to perform the Incremental Backups now.

Bugs Fixed:

• innobackupex is using SHOW MASTER STATUS to obtain binlog file and position. This could trigger a bug if the server being backed up was standalone server (neither master nor slave in replication) and binlog information wasn’t available. Fixed by not creating xtrabackup_binlog_info file when binlog isn’t available. Bug fixed #1168513.
• Percona XtraBackup would leave xbtemp temp files behind due to a typo. Bug fixed #1172016.
• Percona XtraBackup would assume the table has been dropped if the tablespace was renamed after it was scanned by XtraBackup on startup and before XtraBackup attempted to copy it. Bug fixed #1079700.
• Orphaned xtrabackup_pid file left inside tmpdir could cause SST to fail. Fixed by fix checking if xtrabackup_pid file exists once innobackupex starts, and try to remove it or fail if it cannot be removed. Bug fixed #1175860.
• xtrabackup –stats option would not work with server datadir if the server isn’t running and logs were in a separate directory. Bug fixed #1174314.

Other bugs fixed: bug fixed #1166713, bug fixed #1175581, bug fixed #1175318, bug fixed #1175309, bug fixed #1176198, bug fixed #1175566.

Release notes with all the bugfixes for Percona XtraBackup 2.1.1 are available in our online documentation. Bugs can be reported on the launchpad bug tracker.

The post Announcing Percona XtraBackup 2.1.1 GA appeared first on MySQL Performance Blog.

• NoSQL Roadshow, Berlin Germany, May 16, 2013
  
• Talk: NoSQL & MySQL by Mario Beck
• Solutions Libres & Open Source, Paris, France, May 28-29, 2013
• Talk: NoSQL & MySQL by Olivier Zemrag
• Meet MySQL, Oracle Linux and VM team at our booth E40 in the main exhibition hall!
  
• International PHP Conference, Berlin, Germany, June 4-5, 2013
• Talks: MySQL meets NoSQL & An auto-sharding Database Cluster in a Nutshell by Ulf Wendel; Write your own SAPI by Andrey Hristov
• Meet us at our MySQL booth too!
  
• DevConf 2013, Moscow, Russia, June 14-15, 2013
• MySQL team has submitted about 5 talks, however the CfP is still open, so the accepted talks are not yet announced.
  
• Lets meet our technical team at the MySQL booth!
  
• OpenSUSE, Thessaloniki, Greece, July 18-22, 2013
• You can find MySQL team at the MySQL booth as well as find several talks at openSUSE!
  
• Froscon 2013, St. Augustin, Germany, August 24-25, 2013
• Froscon is starting to be a tradition. Same as last year you can find our MySQL staff at the booth and also talking about MySQL news during the conference!
  
• DrupalCon, Prague, Czech Republic, September 23-27, 2013
• We are pleased to invite you to visit our MySQL booth as well as find several MySQL related talks we are planning to submit!

If you are interested, you can find the other conferences we are attending (above EMEA area) at our MySQL Community wikis. We are looking forward seeing and talking to you!

SELECT DATA_FREE FROM INFORMATION_SCHEMA.TABLES WHERE ENGINE='InnoDB' LIMIT 1;

SELECT SUM(DATA_FREE) FROM INFORMATION_SCHEMA.TABLES WHERE ENGINE='InnoDB';

mysql> SELECT TABLE_NAME,DATA_FREE,
    -> udf_fileexists(CONCAT(TABLE_SCHEMA,'/',TABLE_NAME,'.ibd')) AS ibd_file 
    -> FROM information_schema.tables WHERE ENGINE='InnoDB';
+----------------------+-----------+----------+
| TABLE_NAME           | DATA_FREE | ibd_file |
+----------------------+-----------+----------+
| t1                   |   4194304 |        1 |
| t2                   |   4194304 |        1 |
| t3                   |  66060288 |        0 |
| t4                   |  66060288 |        0 |
| innodb_index_stats   |         0 |        1 |
| innodb_table_stats   |         0 |        1 |
| slave_master_info    |         0 |        1 |
| slave_relay_log_info |         0 |        1 |
| slave_worker_info    |         0 |        1 |
+----------------------+-----------+----------+
9 rows in set (0.02 sec)

mysql> SELECT (SELECT DATA_FREE FROM INFORMATION_SCHEMA.TABLES WHERE 
    -> NOT udf_fileexists(CONCAT(TABLE_SCHEMA,'/',TABLE_NAME,'.ibd')) 
    -> AND ENGINE='InnoDB' LIMIT 1) + (SELECT SUM(DATA_FREE) 
    -> FROM INFORMATION_SCHEMA.TABLES WHERE 
    -> udf_fileexists(CONCAT(TABLE_SCHEMA,'/',TABLE_NAME,'.ibd')) 
    -> AND ENGINE='InnoDB') AS TOTAL_DATA_FREE;
+-----------------+
| TOTAL_DATA_FREE |
+-----------------+
|        74448896 |
+-----------------+
1 row in set (0.03 sec)

mysql> SELECT T.TABLE_SCHEMA,T.TABLE_NAME,T.DATA_FREE,ST.SPACE,SD.PATH 
    -> FROM INFORMATION_SCHEMA.TABLES T LEFT JOIN INFORMATION_SCHEMA.INNODB_SYS_TABLES ST 
    -> ON ST.NAME=CONCAT(T.TABLE_SCHEMA,'/',T.TABLE_NAME) 
    -> LEFT JOIN INFORMATION_SCHEMA.INNODB_SYS_DATAFILES SD ON SD.SPACE=ST.SPACE WHERE T.ENGINE='InnoDB';
+---------------------+----------------------+-----------+-------+----------------------------------+
| TABLE_SCHEMA        | TABLE_NAME           | DATA_FREE | SPACE | PATH                             |
+---------------------+----------------------+-----------+-------+----------------------------------+
| mysql               | innodb_table_stats   |         0 |     1 | ./mysql/innodb_table_stats.ibd   |
| mysql               | innodb_index_stats   |         0 |     2 | ./mysql/innodb_index_stats.ibd   |
| mysql               | slave_relay_log_info |         0 |     3 | ./mysql/slave_relay_log_info.ibd |
| mysql               | slave_master_info    |         0 |     4 | ./mysql/slave_master_info.ibd    |
| mysql               | slave_worker_info    |         0 |     5 | ./mysql/slave_worker_info.ibd    |
| innodbfreespacetest | t1                   |   4194304 |     6 | ./innodbfreespacetest/t1.ibd     |
| innodbfreespacetest | t2                   |   4194304 |     7 | ./innodbfreespacetest/t2.ibd     |
| innodbfreespacetest | t3                   |  66060288 |     0 | NULL                             |
| innodbfreespacetest | t4                   |  66060288 |     0 | NULL                             |
+---------------------+----------------------+-----------+-------+----------------------------------+
9 rows in set (0.02 sec)

mysql> SELECT (SELECT SUM(DATA_FREE) FROM INFORMATION_SCHEMA.TABLES T 
    -> LEFT JOIN INFORMATION_SCHEMA.INNODB_SYS_TABLES ST ON 
    -> ST.NAME=CONCAT(T.TABLE_SCHEMA,'/',T.TABLE_NAME) WHERE SPACE<>0) + 
    -> (SELECT DATA_FREE FROM INFORMATION_SCHEMA.TABLES T 
    -> LEFT JOIN INFORMATION_SCHEMA.INNODB_SYS_TABLES ST 
    -> ON ST.NAME=CONCAT(T.TABLE_SCHEMA,'/',T.TABLE_NAME) 
    -> WHERE SPACE=0 LIMIT 1) AS INNODB_FREE_GLOBAL;
+--------------------+
| INNODB_FREE_GLOBAL |
+--------------------+
|           74448896 |
+--------------------+
1 row in set (0.04 sec)

Everything starts with lex.h and sql_yacc.yy in the sql/ dir. lex.h contains all the functions and symbols used to make up the SQL syntax. The sql_yacc.yy file describes the relationships between these symbols and the C functions responsible for executing them. I’m not sure why some symbol definitions end in _SYM and others don’t. Looking in lex.h “FLUSH” is defined as FLUSH_SYM. To see all the places where flush is allowed in the SQL go back to sql_yacc.yy and grep for it.

The first important section looks like this:

/* flush things */                                                                      

flush:                                                                                  
          FLUSH_SYM opt_no_write_to_binlog                                              
          {                                                                             
            LEX *lex=Lex;                                                               
            lex->sql_command= SQLCOM_FLUSH;                                             
            lex->type= 0;                                                               
            lex->no_write_to_binlog= $2;                                                
          }                                                                             
          flush_options                                                                 
          {}                                                                            
        ;

This is where things can get a bit nested and weird. The flush: section is saying that flush can have opt_no_write_to_binlog optionally after it. The first section in curly braces defines the sql command and also sets the flag no_write_to_binlog. SQLCOM_FLUSH is important in the next phase where we get into actual C code.

flush_options used to define all of the possible options for a flush command. Going one step further down flush_options_list basically says that a flush command can contain more than one option.

flush_options_list:
          flush_options_list ',' flush_option
        | flush_option
          {}
        ;

Notice that flush_options_list can contain a flush_options_list. I don’t know the specifics of this but it is the yacc way of saying things can be repeated. In this case the | is saying that there can be multiple flush_option separated by a comma or just one option.

With flush_option: things start to make more sense. This is all of the different types of flush commands. Looking at the first part

flush_option:
          ERROR_SYM LOGS_SYM
          { Lex->type|= REFRESH_ERROR_LOG; }
        | ENGINE_SYM LOGS_SYM
          { Lex->type|= REFRESH_ENGINE_LOG; }
        | GENERAL LOGS_SYM
          { Lex->type|= REFRESH_GENERAL_LOG; }
        | SLOW LOGS_SYM

Reading it in english this is basically saying  ”Error logs OR engine logs OR general logs OR slow logs” Combining this with the previous section allowing multiple flush options this is a valid query:

MariaDB [test]> flush error logs, slow logs;
Query OK, 0 rows affected (0.00 sec)

The flush command is quite a bit improved in MariaDB 10. Comparing this to part of the flush_option: section from MariaDB 5.2 shows how much it has improved:

flush_option:
          table_or_tables
          { Lex->type|= REFRESH_TABLES; }
          opt_table_list {}
        | TABLES WITH READ_SYM LOCK_SYM
          { Lex->type|= REFRESH_TABLES | REFRESH_READ_LOCK; }
        | QUERY_SYM CACHE_SYM
          { Lex->type|= REFRESH_QUERY_CACHE_FREE; }
        | HOSTS_SYM
          { Lex->type|= REFRESH_HOSTS; }
        | PRIVILEGES
          { Lex->type|= REFRESH_GRANT; }
        | LOGS_SYM
          { Lex->type|= REFRESH_LOG; }
        | STATUS_SYM
          { Lex->type|= REFRESH_STATUS; }

In 5.2 the LOGS_SYM is alone which makes flush error logs an invalid query. By scanning the grammar in sql_yacc.yy it is easy to see which syntax is and isn’t supported between versions.

Now that a command of SQLCOM_FLUSH has been specified. The flush_option is passed in via Lex->type. Each option is bitwise ORed into type. It is time to switch over to C++ code and see how these are executed.

sql_parse.cc has a huge switch case statement that contains every possible command type that MySQL can process. For this example look for case SQLCOM_FLUSH: The SQLCOM_FLUSH is the same option from the grammar file. There is basically one important function under this section reload_acl_and_cache().

In newer versions reload_acl_and_cache() is in sql_reload.cc. In older versions it is in sql_parse.cc This function basically checks each type of thing that can be flushed one by one to see if their flag has been ORed into Lex-type which is options here. It then calls the C++ code responsible for carrying out the flushing of that type of object.

Other types of calls can be traced the same way. Most of the token names can be easily grepped from sql_yacc.yy. Most of the show commands are handled similar to flush except when it gets into C++ code they are mapped into a special pair of arrays that tell MySQL how to generate a INFORMATION_SCHEMA table.

For the show commands the C++ code is in the sql_yacc.yy file as a call to prepare_schema_table(). There are two arrays that are important when adding a new table. In MariaDB 10 the first is enum_schema_tables in handler.cc. In older versions this array is in table.h. The other array is ST_SCHEMA_TABLE schema_tables in sql_show.cc.

schema_tables has the important information. Among other things it holds the text name of the table, the fields used to make it, and the function called to generate the data in the table. The fields list usually ends with _fields_info and the function used to create the result set used for the table starts with fill_.

The sad reality is the complexity of the non-obviousness of the bits o the storage engine API you actually care about are documented in ha_ndbcluster.cc, ha_myisam.cc and ha_innodb.cc. If you’re doing something that isn’t already done by one of those three engines: good luck.

Whenever I looked at ha_example.cc I always wished there was something more behind it… basically hoping that InnoDB would get a better and cleaner API with the server and would use that rather than the layering violations it has to do the interesting stuff.

That all being said, as a starting point, it probably helped spawn at least a dozen storage engines.

character_set_client      latin1
character_set_connection  latin1
character_set_database    latin1
character_set_filesystem  binary
character_set_results     latin1
character_set_server      latin1
character_set_system      utf8

I found a workaround (at least as of 5.6.6 and more recent) which gives an indication whether yaSSL or OpenSSL was used.  The Rsa_public_key status variable is explicitly defined only when yaSSL libraries are not used:

#ifndef HAVE_YASSL
  {"Rsa_public_key",           (char*) &show_rsa_public_key, SHOW_FUNC},
#endif

As a result, MySQL Enterprise 5.6.10 (with OpenSSL) has Rsa_public_key status variable:

mysql> select version();
+---------------------------------------+
| version()                             |
+---------------------------------------+
| 5.6.10-enterprise-commercial-advanced |
+---------------------------------------+
1 row in set (0.02 sec)

mysql> show status like '%rsa%';
+----------------+-------+
| Variable_name  | Value |
+----------------+-------+
| Rsa_public_key |       |
+----------------+-------+
1 row in set (0.00 sec)

while MySQL Community 5.6.10 does not:

mysql> select version();
+-----------+
| version() |
+-----------+
| 5.6.10    |
+-----------+
1 row in set (0.00 sec)

mysql> show status like '%rsa%';
Empty set (0.00 sec)

Hopefully that will help others that have a need similar to Daniel and myself.  Hopefully we’ll get a global status variable that makes this indirect method obsolete.

Please see the official invitation in English and Norwegian below.

Invitation to MySQL mini-seminar

Many people are unaware that a great number of the MySQL developers are located in Trondheim, Norway, and that we have a lot of competence we would love to share.

We therefore invite anyone in Trondheim who's interested in MySQL to a mini-seminar located the Oracle's offices on Lade, June 19 at 3 pm.

The agenda will be:

• Presentation: MySQL Query Optimizer: an overview, by Jørgen Løland, MySQL Optimizer team
• Q&A, discussions
• Suggestions for presentations on future mini-seminars
• Food and mingling.

Don't miss this opportunity to meet with other MySQL users and developers behind the worlds most popular open source database system!

Date: June 19
Time: 3pm - 6pm
Location: Oracle, Haakon VIIs g 7b. (across the road from City Lade)
Registration: jorgen.loland@oracle.com
Remember to register so we know how many to expect.

Invitasjon til MySQL mini-seminar

For mange er det ukjent at en stor andel av utviklerne bak MySQL holder til i Trondheim, og vi har mye kompetanse som vi gjerne vil dele.

MySQL-interesserte i Trondheim inviteres derfor til mini-seminar i Oracles lokaler på Lade 19. juni kl 15:00.

Programmet vil bestå av:

• Faglig: MySQL Query Optimizer: an overview, ved Jørgen Løland, MySQL Optimizer team
• Spørsmål og diskusjon
• Forslag til faglig innhold ved framtidige mini-seminarer.
• Mat og mingling.

Ikke gå glipp av denne muligheten til å møte både andre MySQL brukere og utviklere bak verdens mest populære open source databasesystem!

Dato: 19. juni
Tidspunkt: 15:00 - 18:00
Sted: Oracle, Haakon VIIs g 7b. (rett over veien fra City Lade)
Påmelding: jorgen.loland@oracle.com
Husk å melde deg på slik at vi vet omtrent hvor mange vi blir.

A notable fix in Connector/Python v1.0.10 which might interest a few users is adding support for LOAD DATA LOCAL INFILE. It allows you to import CSV using a simple SQL statement.

Please use the MySQL Bugs website to report any problem.

Some useful links:

• Documentation: http://dev.mysql.com/doc/connector-python/en/index.html
• Release Notes: http://dev.mysql.com/doc/relnotes/connector-python/en/index.html
• Downloads: http://dev.mysql.com/downloads/connector/python/#downloads
• Feedback: http://bugs.mysql.com
• Forum: http://forums.mysql.com/list.php?50

Enjoy!

• Choosing the Best Sharding Policy - Doran Levari (ScaleBase, using a video link)
• Performance Monitoring with Statsd and Graphite - Art van Scheppingen (Spil Games)
• Basic MySQL performance tuning for sysadmins - Daniël van Eeden (Snow)

Not very compressible data

Insert performance with no indexes (best to worst)

Query OK, 15000000 rows affected (1 min 20.25 sec)

Query OK, 15000000 rows affected (2 min 5.40 sec)

Query OK, 15000000 rows affected (4 min 3.15 sec)

---TRANSACTION 514, ACTIVE 365 sec inserting
1 lock struct(s), heap size 376, 0 row lock(s), undo log entries 2233162

Test with indexes

alter table insert_test add key(LO_CustKey), add key(LO_OrderDateKey),
add key(LO_SuppKey), add key(LO_PartKey);

Query OK, 15000000 rows affected (2 min 58.42 sec)

Query OK, 15000000 rows affected (4 min 40.05 sec)

Insert performance with more highly compressible data

Query OK, 5000000 rows affected (45.34 sec)

Query OK, 5000000 rows affected (1 min 14.89 sec)

Query OK, 5000000 rows affected (1 min 16.75 sec)

Query OK, 5000000 rows affected (1 min 55.24 sec)

Query OK, 5000000 rows affected (2 min 16.23 sec)

Query OK, 5000000 rows affected (4 min 20.12 sec)

Query OK, 5000000 rows affected (10 min 58.43 sec)

Building index on copy of the 15M row table (best to worst)

alter table sortbuild_test  add key(LO_CustKey), add key(LO_OrderDateKey),  add key(LO_SuppKey), add key(LO_PartKey);

Query OK, 0 rows affected (1 min 40.55 sec)

Query OK, 0 rows affected (2 min 21.93 sec)

Building index on copy of the 5M row table (InnoDB 8k only)

Query OK, 0 rows affected (1 min 14.05 sec)

Data size comparison (smallest disk footprint to largest)

+------------------------------+-------------------+---------+---------+----------+
| table_schema                 | table_name        | data_mb | idx_mb  | sum_size |
+------------------------------+-------------------+---------+---------+----------+
| archive_phrase               | insert_test       |  529.78 |    0.00 |   529.78 |
| archive                      | insert_test       | 1386.51 |    0.00 |  1386.51 |
| innodb_compressed_phrase     | insert_test_8k    | 1601.00 |    0.00 |  1601.00 |
| innodb_compressed_phrase_idx | sortbuild_test_8k | 1601.00 |  151.30 |  1752.30 |
| innodb_compressed_phrase_idx | insert_test_8k    | 1601.00 |  433.83 |  2034.83 |
| tokudb                       | insert_test       |    X    |    X    |     X    |
| tokudb_phrase                | insert_test       |    X    |    X    |     X    |
| tokudb_phrase_idx            | insert_test       |    X    |    X    |     X    |
| innodb                       | insert_test       | 3112.00 |    0.00 |  3112.00 |
| innodb_phrase                | insert_test       | 3202.00 |    0.00 |  3202.00 |
| tokudb_idx                   | insert_test       |    X    |    X    |     X    |
| tokudb_idx                   | sortbuild_test    |    X    |    X    |     X    |
| innodb_phrase_idx            | insert_test       | 3202.00 |  565.03 |  3767.03 |
| innodb_idx                   | sortbuild_test    | 3112.00 |  899.19 |  4011.19 |
| innodb_idx                   | insert_test       | 3112.00 | 1519.00 |  4631.00 |
+------------------------------+-------------------+---------+---------+----------+

CREATE TABLE IF NOT EXISTS insert_test
(
LO_OrderKey bigint not null,
LO_LineNumber tinyint not null,
LO_CustKey int not null,
LO_PartKey int not null,
LO_SuppKey int not null,
LO_OrderDateKey int not null,
LO_OrderPriority varchar(15),
LO_ShipPriority char(1),
LO_Quantity tinyint,
LO_ExtendedPrice decimal,
LO_OrdTotalPrice decimal,
LO_Discount decimal,
LO_Revenue decimal,
LO_SupplyCost decimal,
LO_Tax tinyint,
LO_CommitDateKey int not null,
LO_ShipMode varchar(10),
extra_data TEXT
);

For compression:
ROW_FORMAT=compressed, KEY_BLOCK_SIZE=8

[mysqld]
#datadir=/data/datadirs/tokudb_55
datadir=/data/datadirs/percona_55
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

innodb_buffer_pool_size=16G
innodb_log_file_size=4G
innodb_flush_neighbor_pages=cont
innodb_fast_checksum
innodb_file_per_table
innodb_file_format=barracuda
innodb_buffer_pool_instances=6
innodb_write_io_threads=12
innodb_read_io_threads=12
innodb_flush_method=O_DIRECT
innodb_io_capacity=10000

read_buffer_size=2M
key_buffer_size=32M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

Intel i970-3.2GHz 6 core w/HT (12 threads)
24GB RAM
LSI 9211-8i IT HBA
DB on software RAID 0 - Two Intel 520 SSD, one OCZ vertex.  300GB total.

Facts about Galera replication

There’s a lot of different facts about Galera that come into play here, and it isn’t always obvious how they will affect your database workload.  For example:

• Transaction commit takes approximately the worst packet round trip time (RTT) between any two nodes in your cluster.
• Transaction apply on slave nodes is still asynchronous from client commit (except on the original node where the transaction is committed)
• Galera prevents writing conflicts to these pending transactions while they are inflight in the form of deadlock errors.  (This is actually a form of Eventual Consistency where the client is forced to correct the problem before it can commit.  It is NOT the typical form of Eventual Consistency, known as asynchronous repair, that most people think of).
  

Callaghan’s Law

But what does that all actually mean?  Well, at the Percona Live conference a few weeks ago I heard a great maxim that really helps encapsulate a lot of this information and puts it into context with your application workload:

[In a Galera cluster] a given row can’t be modified more than once per RTT

This was attributed to Mark Callaghan from Facebook by Alexey Yurchenko from Codership at his conference talk.  Henceforth this will be known as “Callaghan’s law” in Galera circles forever, though Mark didn’t immediately recall saying it.

Applied to a standalone Innodb instance

Let’s break it down a bit.  Our unit of locking in Innodb is a single row (well, the PRIMARY KEY index entry for that row).  This means typically on a single Innodb node we can have all sorts modifications floating around as long as they don’t touch the same row.  Row locks are held for modifications until the transaction commits and that takes an fsync to the redo log by default, so applying Callaghan’s law to single-server Innodb, we’d get:

[On a single node Innodb server] a given row can’t be modified more than the time to fsync

You can obviously relax that by simply not fsyncing every transaction (innodb_flush_log_at_trx_commit != 1), or work around it with by fsyncing to memory (Battery or capacitor-backed write cache), etc., but the principle is basically the same.  If we want this transaction to persist after a crash, it has to get to disk.

This has no effect on standard MySQL replication from this instance, since MySQL replication is asynchronous.

What about semi-sync MySQL replication?

It’s actually much worse than Galera.  As I illustrated in a blog post last year, semi-sync must serialize all transactions and wait for them one at a time.  So, Callaghan’s law applied to semi-sync is:

[On a semi-sync replication master] you can’t commit (at all) more than once per RTT. 

Applied to a Galera cluster

In the cluster we’re protecting the data as well, though not by ensuring it goes to disk (though you can do that).  We protect the data by ensuring it gets to every node in the cluster.

But why every node and not just a quorum?  Well, it turns out transaction ordering really, really matters (really!).  By enforcing replication to all nodes, we can (simultaneously) establish global ordering for the transaction, so by the time the original node gets acknowledgement of the transaction back from all the other nodes, a GTID will also (by design) be established.  We’ll never end up with non-deterministic ordering of transactions as a result.

So this brings us back to Callaghan’s law for Galera.  We must have group communication to replicate and establish global ordering for every transaction, and the expense of doing that for Galera is approximately one RTT between the two nodes in the cluster that are furthest apart (regardless of where the commit comes from!).  The least amount of data we can change in Innodb at a time is a single row, so the most any single row can be modified cluster-wide is once per RTT.

What about WAN clusters?

Callaghan’s law applies to WAN clusters as well.  LANs usually have sub-millisecond RTTs.  WANs usually have anywhere from a few ms up to several hundred.  This really will open a large window where rows won’t be able to be updated more than just a few times a second at best.

Some things the rule does not mean on Galera

• It does NOT mean you can’t modify different rows simultaneously.  You can.
• It does NOT mean you can’t modify data on multiple cluster nodes simultaneously.  You can.
• It does NOT set an lower bound on performance, only a upper bound.  The best performance you can expect is modifying a given row once per RTT, it could get slower if apply times start to lag.

So what about my application?

Think about your workload.  How frequently do you update any given row?  We call rows that are updated heavily “hotspots“.

Examples of hotspots

Example 1: Your application is an online game and you keep track of global achievement statistics in a single table with a row for each stat; there are just a few hundred rows.  When a player makes an achievement, your application updates this table with a statement like this:

UPDATE achievements SET count = count + 1 where achievement = 'killed_troll';

How many players might accomplish this achievement at the same time?

Example 2: You have users and groups in your application.  These are maintained in separate tables and there also exists a users_groups table to define the relationship between them.  When someone joins a group, you run a transaction that adds the relationship row to users_groups, but also updates groups with some metadata:

BEGIN;
INSERT INTO users_groups (user_id, group_id) VALUES (100, 1);
UPDATE groups SET last_joined=NOW(), last_user_id=100 WHERE id=1;
COMMIT;

How often might multiple users join the same group?

Results

In both of the above examples you can imagine plenty of concurrent clients attempting to modify the same record at once.  But what will actually happen to the clients who try to update the same row within the same RTT?  This depends on which node in the cluster the writes are coming from:

From the same node: This will behave just like standard Innodb.  The first transaction will acquire the necessary row locks while it commits (which will take the 1 RTT).  The other transactions will lock wait until the lock(s) they need are available.  The application just waits in those cases.

From other nodes: First to commit wins.  The others that try to commit AFTER the first and while the first is still in the local apply queue on their nodes will get a deadlock error.

So, the best case (which may not be best for your application database throughput) will be more write latency into the cluster.  The worst case is that your transactions won’t even commit and you have to take some action you normally wouldn’t have had to do.

Workarounds

If your hotspots were really bad in standalone Innodb, you might consider relaxing the fsync:  set innodb_flush_log_at_trx_commit to something besides 1 and suddenly you can update much faster.  I see this tuning very frequently for “performance” reasons when data durability isn’t as crucial.  This is fine as long as you weigh both options carefully.

But in Galera you cannot relax synchronous replication.  You can’t change the law, you can only adapt around it, but how might you do that ?

Write to one node

If your issue is really the deadlock errors and not so much the waiting, you could simply send all your writes to one node.  This should prevent the deadlock errors, but will not change the lock waiting that your application will need to do for hotspots.

wsrep_retry_autocommit

If your hotspots are all updates with autocommits, you can rely on wsrep_retry_autocommit to auto-retry the transactions for you.  However, each autocommit is retried only the number of times specified by this variable (default is 1 retry).  This means more waiting, and after the limit is exceeded you will still get the deadlock error.

This is not implemented for full BEGIN … COMMIT multi-statement transactions since it cannot be assumed that those are not applying application logic in between the statements that is not safe to retry after the database state changes.

retry deadlocks

Now we start to get into (*gasp*) territory where your application needs to be modified.  Generally if you use Innodb, you should be able to handle deadlock errors in your application.  Raise your hands if your application has that logic (I usually get less than 5 people who do out of 100).

But, what to do?  Retrying automatically, or giving your end user a chance to retry manually are typical answers.  However, this means more latency waiting for a write to go through, and possibly some poor user experience.

batch writes

Instead of updating global counters one at a time (from Example 1, above), how about maintaining the counter in memcache or redis and only flushing to the database periodically?

if( $last_count % 100 == 0 ) {
  $db->do( "UPDATE achievements SET count = $last_count where achievement = 'killed_troll'";
}

change your schema

In Example 2, above, how above moving the ‘joined’ column to the users_groups table so we don’t need to update the parent group row so often?

INSERT INTO users_groups (user_id, group_id, joined) VALUES (100, 1, NOW());

Conclusion

Choosing a system to replicate your data to a distributed system requires tradeoffs.  Most of us are used to the tradeoffs we take when deploying conventional stand-alone MySQL Innodb with asynchronous slaves.  We may not think about the tradeoffs, but we’re making them (anyone obsessively testing slave position to ensure it’s caught up with the master?).

Synchronous replication with PXC and Galera is no different in that there are trade-offs, they just aren’t what we commonly expect.

If Callaghan’s law is going to cause you trouble and you are not prepared to adapt to work with it, PXC/Galera Synchronous replication is probably not right for you.

The post Is Synchronous Replication right for your app? appeared first on MySQL Performance Blog.

Here’s a sample, which I’ll run in both 5.5 and 5.6 and show the resulting GQL:

mysql> SELECT 1;
+---+
| 1 |
+---+
| 1 |
+---+
1 row in set (0.00 sec)

mysql> SELECT NOTHING();
ERROR 1305 (42000): FUNCTION NOTHING does not exist
mysql> SELECT 2;
+---+
| 2 |
+---+
| 2 |
+---+
1 row in set (0.00 sec)

In 5.5, this produces the following in the general query log:

130513 18:26:34        1 Query    SELECT 1
130513 18:26:40        1 Query    SELECT NOTHING()
130513 18:26:44        1 Query    SELECT 2

In 5.6, the same produces the following:

130425 21:53:25        1 Query    SELECT 1
130425 21:53:35        1 Query    SELECT 2

The behavior hasn’t changed between 5.5 and 5.6 with respect to successfully-parsed, but unauthorized statements.  With the limited-privilege anonymous user account, I issued the following against both 5.5 and 5.6 servers:

mysql> SHOW GRANTS;
+--------------------------------------+
| Grants for @localhost                |
+--------------------------------------+
| GRANT USAGE ON *.* TO ''@'localhost' |
+--------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT * FROM mysql.user;
ERROR 1142 (42000): SELECT command denied to user ''@'localhost' for table 'user'

The general query log for both 5.5 and 5.6 recorded the attempt to SELECT from mysql.user system table:

130513 18:31:10        3 Query    SHOW GRANTS
130513 18:31:11        3 Query    SELECT * FROM mysql.user

The documentation doesn’t explicitly note this behavior change (I filed Bug#68937 to include this in the manual) – it talks about the password-masking feature which triggered this behavioral change, though (and this page also documents which statements are rewritten).  In order to mask passwords in log files, the log entries have to be written after they are parsed.  When I issue the following statement in 5.6, the password is masked in the general query log:

mysql> SET PASSWORD = PASSWORD('test');
Query OK, 0 rows affected (0.00 sec)

Here’s the corresponding general query log entry:

130513 18:45:59        2 Query    SET PASSWORD FOR `root`=<secret>

That’s much appreciated behavior – there’s typically no reason to expose passwords in logs.  For those who do need this temporarily for diagnostic purposes, there’s a –log-raw option which logs before parsing, just like in 5.5.  This means that plain-text passwords as well as statements with syntax errors get logged.  Here’s the result in 5.6 with –log-raw enabled:

130513 18:43:10        1 Query    SELECT NOTHING()

Unfortunately, there’s no status variable to tell a DBA whether or not they are protected by the new 5.6 behavior, or whether the running server has been started with –log-raw to override it and is still logging plain-text passwords.  I filed Bug#68936 to address that.  I would also love to allow users (with appropriate permissions) the ability to change this configuration option without restart of MySQL Server, but it’s probably not something that will need – or want – to be changed in a production environment where downtime is critical.

I’m happy to see plain-text passwords removed from logs in 5.6, and hope that this post helps clarify associated behavioral changes related to the general query log in 5.6.

Finding a SATA III controller with more than one PCI-e lane

Strange Windows 7 Professional 64 bit performance

Linux and the case of the mysterious hanging mkfs.ext4

Fixing the mkfs.ext4 issue

To place the card in RESET MODE:
DO NOT REBOOT WITH THE CARD IN THIS STATE OR YOU WILL HAVE TO RMA THE CARD!!!!!
C:\sas2flsh.exe -o -e 6

Now you must flash the new firmware AND bios:
C:\>sas2flsh.exe -o -f FIRMWARE.bin -b BIOS.rom 

It is safe to reboot after flashing

Getting rid of raid (flash to IT mode)

Final setup

You can actually easily beat ARCHIVE for INSERT speed with a non-indexed MyISAM table, and with things like TokuDB around you can probably get pretty close to compression while at the same time having these things known as “indexes”.

ARCHIVE for a long time held this niche though and was widely and quietly used (and likely still is). It has the great benefit of being fairly lightweight – it’s only about 2500 lines of code (1130 if you exclude azio.c, the slightly modified gzio.c from zlib).

It also use the table discovery mechanism that NDB uses. If you remove the FRM file for an ARCHIVE table, the ARCHIVE storage engine will extract the copy it keeps to replace it. You can also do consistent backups with ARCHIVE as it’s an append-only engine. The ARCHIVE engine was certainly the simplest example code of this and a few other storage engine API things.

I’d love to see someone compare storage space and performance of ARCHIVE against TokuDB and InnoDB (hint hint, the Internet should solve this for me).

I’ll give an overview on how Fractal Tree® indexes work, and then get into specific product features that Fractal Trees enable in MySQL and MongoDB.  Some benchmarking and customer use-cases will be discussed, but my intent is for this to be a deep technical dive.  Several Tokutek Engineers will also be on hand, so bring any questions you’ve got.

I hope to see you there!