Showing entries 31 to 40 of 80
« 10 Newer Entries | 10 Older Entries »
Displaying posts with tag: Testing (reset)
Testing Xeround’s database as a service

So while I was at the MySQL UC, The Xeround database came to my attention.  It bills itself as database as a service for MySQL systems and a seamless replacement for standard MySQL.

Of course, since I am a QA Engineer, I could not resist the urge to try to break it >:)  As my friend and former MySQL colleage, Kostja says, “QA Engineers are a unique breed…they like to push all the buttons” : )  I would say that the QA mindset goes a bit further than that, but it is something I will delve into in another post.  I will only say that there is a reason that Microsoft recognizes QA software engineering as a distinct and specialized discipline.

So, let’s get back to Xeround.  It was the first database as a service that caught my eye and I just had to test it!  They are currently …

[Read more]
More on kewpie (the query probulator)

My presentation from the MySQL UC didn’t give a lot of detail on the actual tool I have hacked up, nor did it go into how to play with it / try it out.  I figured I should rectify that (at least one person seemed interested in trying it out <g>)

To begin with, you should have the random query generator installed (see the docs for handling that).  Besides being *the* cutting edge, production-ready testing tool in the open-source dbms world, it comes with a handy data generator.

One of the key features of kewpie, is that it can easily generate test queries against any test bed.  A standard randgen practice is to develop grammars and gendata files (which generates a user-specified …

[Read more]
It's all about embedded (and testing)

In his recent "Test Coverage for your Enterprise Beans" blog post Markus Eisele starts by discussing the differences between the GlassFish Embedded API, the standard EJB 3.1 API, and the Maven GlassFish Plugin. He then goes on to describe his Eclipse + Eclemma plugin + GlassFish Embedded 3.1 + Maven configuration. He also discusses Global JNDI names (new in EJB 3.1) and the use of ANT with cases when you'll need in-place instrumentation of your code for proper code coverage execution.

[Read more]
It's all about embedded (and testing)

In his recent "Test Coverage for your Enterprise Beans" blog post Markus Eisele starts by discussing the differences between the GlassFish Embedded API, the standard EJB 3.1 API, and the Maven GlassFish Plugin. He then goes on to describe his Eclipse + Eclemma plugin + GlassFish Embedded 3.1 + Maven configuration. He also discusses Global JNDI names (new in EJB 3.1) and the use of ANT with cases when you'll need in-place instrumentation of your code for proper code coverage execution.

[Read more]
Drizzle7 GA – testing summary

With the HUGE news of Drizzle’s first GA, I thought it appropriate that I spend some time discussing the testing that has gone into this release.

I tend to agree with Stewart’s assessment of our quality – it is solid and I think that you will find yourself pleasantly surprised and not at all angry when using it, but it is always in the eye of the user ; )  With that said, I did want to highlight some areas of interest.

To begin with, as we are a fork of MySQL, the bulk of our test suite comes directly from there as well.  Most of the standard mysql-test-run tests that are used to validate MySQL are also used for Drizzle.  All of the basics like …

[Read more]
Quick benchmarking trick

I have been doing quite a lot of benchmarking recently.
I needed to find a safe way of measuring the time spend by the database doing a long task, like catching up on a huge backlog of accumulated replication updates. The problem with measuring this event is that I can record when it starts, but I can't easily detect when it finishes. My initial approach was to monitor the database and count the tables rows to see when the task was done, but I ended up affecting the task performance with my additional queries. So I thought of another method.
Since I had control on what was sent from the master to the slave, I used the following:
The initial time is calculated as the minimum creation time of the databases that I know are created during the exercise. Let's say that I had 5 databases named from db1 to db5:

set @START = (select min(create_time) from information_schema.tables where table_schema like "db%")

[Read more]
dbqp / randgen integration…huzzah!

What is the big deal, you may ask?  Well, read on and all shall be revealed, intrepid reader ; )
As I mentioned an earlier post, our new test-runner – dbqp – allows us to define testing ‘modes’ which all utilize the same system and server management code.

One only has to define a testManager (what does a test look like / how to organize tests) and a testExecutor (how to execute / evaluate a test).  The aim is for dbqp to be a one-stop shop for test execution and to provide a clean and simple way to manage and expand this.

I have just added –mode=randgen to the test-runner.  The random query generator is a significant part of Drizzle’s testing strategy and we use a large number of tests with the …

[Read more]
Automation of functional testing of a key/value storage

In my previous post I described what it took to add SQL support and a simple command line client to a NoSQL storage. However, I needed not just ad-hoc testing with a client, but a framework to automatically run and manage many tests.
I expect that automated tests are easy to understand, extend, and maintain. When a test breaks, finding and debugging what broke should be easy. Such qualities can not be met in a heterogeneous test environment. Rather, it would be best if some common language and toolkit was used. It's easiest for all when a failing test can be run directly under a debugger.
In MySQL, this task is solved with a combination of 'mysqltest' client-side testing tool and 'mysql-test-run', an automation environment for functional tests.
'mysqltest' is …

[Read more]
Functional testing of Tarantool

Tarantool offers clients a simple binary protocol with basic data manipulation commands - GET, PUT, SET, DELETE. All administrative commands, however, must be sent in textual form to a separate, administative port. A separate port is useful as long as there is no authentication support. Examples of administrative commands are 'SAVE SNAPSHOT' or 'SHOW STAT'.
The whole thing had a few functional tests, but all of them required to be run manually. If you're not the one who wrote it, you probably wouldn't know how to run it.
I was looking for something that would be easy to write and easy to run. It needed to be a lingua franca of testing, used both by developers and quality assurance engineers. It would also be nice to be able to easily express in the new framework test cases for discovered bugs.
Of course, for an ex-SQL geek, SQL looked very much like the lingua …

[Read more]
Bringing RDBMS testing methods to a key/value storage project

What I'd like to describe is how I tried to bring things that are good about mysql-test-run, mysqltest and pushbuild, to the open source project I'm currently working on, Tarantool.
Since names such as mysql-test-run, mysqltest, or pushbuild tell little to those who don't know how testing is done at MySQL, I'll make a series of blog posts and try explaining the elephant one piece at a time.
In a nutshell, there is a collection of tools that enable [open source] projects to do development in a "civilized" form. Some projects only use select pieces of the puzzle, but the best effects are, in my view, achieved when all pieces are made to …

[Read more]
Showing entries 31 to 40 of 80
« 10 Newer Entries | 10 Older Entries »