Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 62 Next 30 Older Entries

Displaying posts with tag: Testing (reset)

Testing that all projects need
+1 Vote Up -0Vote Down

Today, I was reminded of a Jim Starkey quote on the Random Query Generator:

“The Colonoscopy of Database Software”
- Jim Starkey

If your project does not have something that you can adapt that quote to, odds are your testing is inadequate.

How Tokutek uses the Random Query Generator framework to test TokuDB
+0 Vote Up -0Vote Down

During a typical release cycle for TokuDB at Tokutek, we spend time qualifying and hardening the product using numerous tools.  For example, we run stress and unit tests directly on the Fractal Tree indexes, MySQL Test Runner (MTR) tests on the storage engine as well as numerous performance benchmarks to prevent regressions. In addition, we have recently been implementing the Random Query Generator (RQG) framework internally here at Tokutek to more exhaustively stress TokuDB.  My name is Joel Epstein and I am a Quality Assurance Engineer here at Tokutek who has been integrating RQG into the overall test plan strategy.


  [Read more...]
External bug reports #2: Build your portfolio
+2 Vote Up -0Vote Down

While user bug reports are the most important ones, there is a category of external reporters which I historically have a special interest in and great expectations for: entry-level testers. I was one, trained some, interviewed many, had a few hired, and have always wanted someone to wake them up and get going before it’s too late.

There is no secret that quality control is not as glamourous as other IT specialities, and there are no famous (or maybe any) student programs for testers. Usually people come into testing because it is deceptively open for newbies, planning to obtain a few points for a CV and switch either to development or to project management as soon as they can. Most do, a few stay.

It creates a vicious cycle. Since this is a

  [Read more...]
Unittesting your indexes
+1 Vote Up -0Vote Down
During FOSDEM PGDay I watched the "Indexes: The neglected performance all-rounder" talk by Markus Winand. Both his talk and the "SQL Performance Explained" book (which is also available online) are great.

The conclusion of the talk is that we should put more effort in carefully designing indexes. But how can we make sure the indexes are really used now and in the future? We need to write some tests for it.

So I wrote a small Python script to test index usage per query. This uses the JSON explain format available in MySQL 5.6. It's just a proof-of-concept so don't expect too much of it yet (but please sent pull requests!).

A short example:

import indextest

  [Read more...]
MariaDB patches for Random Query Generator
+3 Vote Up -0Vote Down

My fellow testers and others who run RQG tests on MySQL flavors might be interested in some additions that are being used for MariaDB testing. While none of them is a major breakthrough, maybe they will make somebody’s life a little easier.

RQG Introduction

A quick introduction for those who have never heard of RQG, but are still curious what this blog post is about.

RQG stands for Random Query Generator, also known as randgen — an open-source product, available under the GPL v2 license. Quoting its home page on Launchpad, it is a “pseudo-random data and query generator that can be used to test any Perl DBI, JDBC or ODBC-compatible SQL server, in particular MySQL, but also JavaDB and PostgreSQL”.

The framework was created by my former

  [Read more...]
MySQL Sandbox supports latest MySQL releases, has more metadata and docs
+2 Vote Up -0Vote Down

MySQL Sandbox has been updated again. The latest version is 3.0.38, which was just released. There were four releases in the space of one week, and this last one is just a polished edition.

Cherry-picking from the Change log:

  • Added option --bind_address to complement the effects of --remote_access;
  • The script 'enable_gtid' (for MySQL 5.6 +) now is durable. Previously the changes did not survive a restart.
  • Now you can install MariaDB with its bizarre version '10.0'
  • It also works well with MySQL 5.7. A bug prevented the creation of 'enable_gtid', but it is

  [Read more...]
Get to know the Random Query Generator
Employee +2 Vote Up -0Vote Down

In the MySQL QA teams in Oracle we have been using a tool called the Random Query Generator (or "RQG" for short) for some time now. The main RQG testing effort has been on new server development, including regression testing. The recent MySQL 5.6 GA is a result of a couple of years of hard work, including a lot of RQG testing and bugfixing as a result of that.

I can easily say that the RQG has helped making MySQL 5.6 a better release than it would otherwise be. It is of course not our only testing tool, but there are plenty of bugs this tool has uncovered that would likely not have been seen in our other testing. Such issues include:

  • crashes and asserts
  • memory management bugs (valgrind errors)
  • wrong

  [Read more...]
Packages to get MariaDB and tests up and running
+0 Vote Up -0Vote Down


It’s often pain to guess package names when you need to install stuff on, lets say, CentOS. So there is a list, although maybe not full, of what I needed to get another VM build and run MariaDB server and to execute at least some tests on it (all done via yum install):


Same in one line, for lazy me:
sudo yum install cmake gcc ncurses-devel bison g++ gcc-c++ aclocal automake libtool perl-DBD-MySQL gdb libaio-devel openssl-devel

To install bzr:

su -c ‘rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm’
(check the architecture)

and then can use yum

  [Read more...]
SkySQL and MariaDB Working Together to Keep MySQL an Open Ecosystem
+8 Vote Up -9Vote Down

I pledged, in my first post last month, that SkySQL will do its part to promote the best of MySQL and its community. Given the recent discovery that Oracle is no longer publishing test cases for bug fixes, and the dialogue surrounding it, it feels like the right time to share my thoughts on what this means to the open source collective, and what we can do – and are doing – about it.

read more

New home for Random Query Generator docs
Employee +0 Vote Up -0Vote Down
One of the tools we use to qualify MySQL Server (http://www.mysql.com/downloads/" target="_blank) is the Random Query Generator (RQG for short). This is an SQL generator (and more) to test database systems such as MySQL (http://www.mysql.com/" target="_blank). I plan to post more on that later.

The RQG is an open source tool written in Perl, and is available on Launchpad. Up until now the RQG documentation has been part of the MySQL Forge wiki. As you may have seen from this blog post, MySQL Forge is going away soon (August 1st). Unlike other parts of the Forge wiki, the Random Query Generator documentation has moved to github, and will continue to be

  [Read more...]
Devops in Munich
+0 Vote Up -1Vote Down

Devopsdays Mountainview sold out in a short 3 hours .. but there's other events that will breath devops this summer.
DrupalCon in Munich will be one of them ..

Some of you might have noticed that I`m cochairing the devops track for DrupalCon Munich,
The CFP is open till the 11th of this month and we are still actively looking for speakers.

We're trying to bridge the gap between drupal developers and the people that put their code to production, at scale.
But also enhancing the knowledge of infrastructure components Drupal developers depend on.

We're looking for talks both on culture (both success stories and failure) , automation,
specifically looking for people talking about drupal deployments , eg using tools like Capistrano, Chef, Puppet,
We want to hear where Continuous Integration fits in your deployment , do you do Continuous Delivery of a

  [Read more...]
Some lessons from MySQL Conference 2012
+9 Vote Up -0Vote Down

The Percona Live MySQL Conference and Expo 2012 is over. Together with the SkySQL solutions day, it has kept me occupied for 4 full days, from early morning to late at night.

I have to say that I am pleased. The quality of the organization was very high, with a very good lineup of speakers and an excellent technical support.

As usual, I have learned a lot during this week, either directly, by attending talks, or indirectly, by meeting people who told me what was juicy at the talks that I had missed. And I have met new interesting people, and caught up with the people that I know already.

This conference was particularly intense also because I got myself involved in 5 talks, which was probably more than I should have. How did

  [Read more...]
dbqp being renamed
+2 Vote Up -1Vote Down

One of the best things that can happen to a piece of software is for people to actually use it.

I’ve been fortunate enough to have received feedback on the tool from several members of both the Percona and Drizzle teams.  The most common and strongly emphasized comments were in regards to what a terrible, terrible name dbqp really is in terms of saying, seeing, and typing it ; )

As that isn’t something that can be disputed (it’s really annoying to use in conversations *and* to type several dozen times a day), the project has been renamed to kewpie.  For those that follow such things, I did present on

  [Read more...]
dbqp and Xtrabackup testing
+2 Vote Up -1Vote Down

So I’m back from the Percona dev team’s recent meeting.  While there, we spent a fair bit of time discussing Xtrabackup development.  One of our challenges is that as we add richer features to the tool, we need equivalent testing capabilities.  However, it seems a constant in the MySQL world that available QA tools often leave something to be desired.  The randgen is a literal wonder-tool for database testing, but it is also occasionally frustrating / doesn’t scratch every testing itch.  It is based on technology SQL Server was using in 1998 (MySQL began using it in ~2007, IIRC).  So this is no knock, it is merely meant to be an example of a poor QA engineer’s frustrations ; )  While the current

  [Read more...]
Drizzle / dbqp updates
+1 Vote Up -0Vote Down

Just wanted to blog about some of the latest updates to dbqp.  We just merged some interesting changes into Drizzle (just in time for the impending Fremont beta).  In additional to general code cleanup / reorganization, we have the following goodies:

Randgen in the Drizzle tree

One of the biggest things is that the random query generator (aka randgen) is now part of the Drizzle tree.  While I did some of the work here, the major drivers of this happening were Brian and Stewart:

  • Brian makes a fair argument that the easier / more convenient it is to run a test, the greater the likelihood of it being
  •   [Read more...]
    Drizzle testing – now with more server stressing goodness!
    +4 Vote Up -0Vote Down

    One of the long term testing goals for Drizzle is to move all of our test logic directly in-tree.  Currently, we use a system called drizzle-automation to execute a variety of tests for our staging branch.  This is the final set of tests patches must pass before being allowed to merge into Drizzle trunk and includes things like sysbench, dbt2, the randgen, etc.  With the development of dbqp, we can now move this testing logic directly into the tree (and even move some of the testing tools there as well).  Of course, I’ve rambled on about this before, but I personally think it is cool and useful ; )  However enough of the sales pitch, on to the new modes!

    sysbench mode

    With but a simple incantation of ./dbqp –mode=sysbench [--suite=readonly|readwrite], you too can invoke the mighty sysbench configurations that we use to ensure

      [Read more...]
    New dbqp feature – using pre-created datadirs for tests
    +2 Vote Up -0Vote Down

    Why would one want to do this, you may ask?  Well, for starters, it makes a great ‘canary-in-the-coal-mine‘ in regards to backwards compatibility!

    For Drizzle, we’ve created some tables (via the randgen’s data generator if you are curious), saved a copy of the datadir, and then created a test case that uses said datadir for the test server.  The test executes some simple SQL queries to make sure we can read the tables properly.  This way, if we ever do something to either the server or .dfe format (data format exchange – had a most enlightening conversation with the team about this

      [Read more...]
    Testing Xeround’s database as a service
    +3 Vote Up -0Vote Down

    So while I was at the MySQL UC, The Xeround database came to my attention.  It bills itself as database as a service for MySQL systems and a seamless replacement for standard MySQL.

    Of course, since I am a QA Engineer, I could not resist the urge to try to break it >:)  As my friend and former MySQL colleage, Kostja says, “QA Engineers are a unique breed…they like to push all the buttons” : )  I would say that the QA mindset goes a bit further than that, but it is something I will delve into in another post.  I will only say that there is a reason that Microsoft recognizes QA software engineering as a distinct and specialized discipline.

    So, let’s get back to Xeround.  It was the first database as a service that caught my eye and I just had to test it!  They are currently offering a

      [Read more...]
    More on kewpie (the query probulator)
    +0 Vote Up -0Vote Down

    My presentation from the MySQL UC didn’t give a lot of detail on the actual tool I have hacked up, nor did it go into how to play with it / try it out.  I figured I should rectify that (at least one person seemed interested in trying it out <g>)

    To begin with, you should have the random query generator installed (see the docs for handling that).  Besides being *the* cutting edge, production-ready testing tool in the open-source dbms world, it comes with a handy data generator.

    One of the key features of kewpie, is that it can easily generate test queries against any test bed.  A standard randgen practice is to develop grammars and gendata files (which generates a user-specified test-bed) that are designed to work

      [Read more...]
    It's all about embedded (and testing)
    Employee_Team +0 Vote Up -0Vote Down

    In his recent "Test Coverage for your Enterprise Beans" blog post Markus Eisele starts by discussing the differences between the GlassFish Embedded API, the standard EJB 3.1 API, and the Maven GlassFish Plugin. He then goes on to describe his Eclipse + Eclemma plugin + GlassFish Embedded 3.1 + Maven configuration. He also discusses Global JNDI names (new in EJB 3.1) and the use of ANT with cases when you'll need in-place instrumentation of your code for proper code coverage execution.

      [Read more...]
    It's all about embedded (and testing)
    Employee_Team +0 Vote Up -0Vote Down

    In his recent "Test Coverage for your Enterprise Beans" blog post Markus Eisele starts by discussing the differences between the GlassFish Embedded API, the standard EJB 3.1 API, and the Maven GlassFish Plugin. He then goes on to describe his Eclipse + Eclemma plugin + GlassFish Embedded 3.1 + Maven configuration. He also discusses Global JNDI names (new in EJB 3.1) and the use of ANT with cases when you'll need in-place instrumentation of your code for proper code coverage execution.

      [Read more...]
    Drizzle7 GA – testing summary
    +0 Vote Up -0Vote Down

    With the HUGE news of Drizzle’s first GA, I thought it appropriate that I spend some time discussing the testing that has gone into this release.

    I tend to agree with Stewart’s assessment of our quality – it is solid and I think that you will find yourself pleasantly surprised and not at all angry when using it, but it is always in the eye of the user ; )  With that said, I did want to highlight some areas of interest.

    To begin with, as we are a fork of MySQL, the bulk of our test suite comes directly from there as well.  Most of the standard mysql-test-run tests that are used to validate MySQL are also used for Drizzle.  All of the basics like creating tables, inserting data,

      [Read more...]
    Quick benchmarking trick
    +3 Vote Up -0Vote Down
    I have been doing quite a lot of benchmarking recently.
    I needed to find a safe way of measuring the time spend by the database doing a long task, like catching up on a huge backlog of accumulated replication updates. The problem with measuring this event is that I can record when it starts, but I can't easily detect when it finishes. My initial approach was to monitor the database and count the tables rows to see when the task was done, but I ended up affecting the task performance with my additional queries. So I thought of another method.
    Since I had control on what was sent from the master to the slave, I used the following:
    The initial time is calculated as the minimum creation time of the databases that I know are created during the exercise. Let's say that I had 5 databases named from db1 to db5:
    set @START = (select min(create_time) from

      [Read more...]
    dbqp / randgen integration…huzzah!
    +0 Vote Up -0Vote Down

    What is the big deal, you may ask?  Well, read on and all shall be revealed, intrepid reader ; )
    As I mentioned an earlier post, our new test-runner – dbqp – allows us to define testing ‘modes’ which all utilize the same system and server management code.

    One only has to define a testManager (what does a test look like / how to organize tests) and a testExecutor (how to execute / evaluate a test).  The aim is for dbqp to be a one-stop shop for test execution and to provide a clean and simple way to manage and expand this.

    I have just added –mode=randgen to the test-runner.  The random query generator is a significant part of Drizzle’s testing strategy and we use a large number of tests with the tool. 

      [Read more...]
    How MySQL Workbench breaks itself
    +6 Vote Up -0Vote Down
    Once upon a time, there was a policy in MySQL not to add new features after the beta stage.
    To my surprise, MySQL Workbench 5.2.30 introduces a new feature, the query formatter. I gave it a try. The results are not extremely encouraging. Granted, it's a plugin and not a feature in the core application, but nonetheless one would expect something more stable in a GA release, especially since the plugin features are displayed in the main menu, and unless you have read the announcement, you couldn't easily tell the core from the plugins.
    This is what I have got in just a few minutes:

    Bug #58356: beautify function fails on CREATE TABLE
    Bug #58357: beutify function erases statement on CREATE

      [Read more...]
    Testing MySQL 5.5 semi-synchronous replication
    +12 Vote Up -0Vote Down
    A few days ago I saw an article about Semi-Synchronous Replication in MySQL 5.5. It asks questions, and doesn't give answers beyond gut feeling. So I thought that I would do some practical testing of this new feature.
    Before we go that way, though, let's revisit the theory.

    How semi-synchronous replication works

    Figure 1. A transaction with regular replication
    With regular replication, you send a transaction to the master (1). When the COMMIT is received, the master executes it (2), and if successful it logs the transaction to the binary log (3). The the master answers the client request (4) with a successful

      [Read more...]
    Slapping MySQL-Proxy
    +0 Vote Up -0Vote Down

    I have old applications that need to read (and write) MyISAM tables that themselves receive lots of bulk updates. Time to try MySQL-Proxy.

    MySQL Proxy is light on documentation and very few people written anything about working.   Most of what I have read says MySQL-Proxy is not ready for prim time.  I have hope so I had to give it a try.

    I started with thee VMware servers. I setup one master and two read only slaves.   I tested the replication with mysqlslap from another independent server and it worked fine.  The slave never ran more then a second behind.

    I downloaded

      [Read more...]
    Performance gain of MySQL 5.1 InnoDB plugin
    +6 Vote Up -0Vote Down
    You know already that InnoDB in MySQL 5.5 has great improvements in performance and scalability. You will have to wait a few months for that, though, because MySQL 5.5 is not GA yet.
    But if you need some extra performance in MySQL 5.1, you may want to use the Innodb Plugin instead of the built-in one. As of version 5.1.47, the Innodb plugin is of GA quality, and it comes with a good out-of-the-box improvement compared to the built-in engine.
    To test my assumptions, I used one of my test Linux servers to perform a sysbench on 5.0.91, 5.1.47 built-in and plugin, and 5.5.4. The MySQL servers

      [Read more...]
    MySQL Sandbox now with plugins, more tests, instrumentation
    +3 Vote Up -0Vote Down
    The latest release of MySQL Sandbox, 3.0.12, has integrated plugin installation features, as mentioned in my previous post.
    Not only that. This version has also more tests, fixes a couple of bugs, and introduces basic instrumentation. Now each script released with MySQL Sandbox, and every one that the Sandbox itself installs, can leave a trail in a file.

    Let's start with the plugin. The documentation has been updated to cover this new feature. And 27 new tests give

      [Read more...]
    Down the rabbit hole
    +0 Vote Up -0Vote Down

    Generally I avoid going down rabbit holes but today I decided to see how deep a particular testing rabbit hole went. This post is a third in what seems be a continuing series of programming anecdotes. It’s not particularly MySQL-related so you can stop reading here unless you grok code stuff.

    Before beginning work on issue 720 I ran the mk-table-checksum test suite to make sure it was in working order. No sense writing new tests and code when the old tests and code aren’t reliable. I actually made one seemingly innocuous change to the test suite in preparation for the issue: I changed the –replicate checksum table from MyISAM to InnoDB.

    Surprisingly, the test suite proved unstable. Random tests would fail at random times. Some

      [Read more...]
    Showing entries 1 to 30 of 62 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.