We have just officially released the WB 5.0 RC3 build and are planning the GA build to happen soon. One might ask, what is our criteria to call something GA? Well, it means that there must not be any known and verified P1 (crashing) and P2 (very serious bug with no workaround) bugs. Does it means that there are no bugs left or that we have implemented every feature request? No.

Therefore our efforts will not stop after the GA build. We still plan to get a new release out every 3rd week including all fixes and improvements that are necessary. This is a first list of things we are planning to release in a future GA release.

• Bug fixes
  Most important are bug fixes of course. Please keep reporting bugs, you did a great job in the past - and if you do so, we will keep closing those bug reports as fast as we can.

• Enable connection-end points dragging/reordering
  This has remained one major …

From time to time it is necessary to get a bigger picture of your project in terms of size, test coverage, code vs. comments ratio and others (aka metrics). While it is quite difficult to find a good (and free) code coverage tool for C++ there is already a very nice tool package called DPack, which is not only freely available but also gives us some very useful additions in our IDE. One of those features is code statistics.

(more…)

For all new contributors devoted to quality, FAQ was slightly updated with new section.

Last post was about Rainbow testing. So that's old news, and I lost count of the number of bugs it found, and have been fixed. Probably 30+ crashes.

Now, I have to put together all these odds & ends of code I wrote disparately over the last 18 months into a single collection to enable complete end-to-end testing.

I have now got roughly the following:

o) random table maker
o) query maker based on any tables, (using predefined rules)
o) data generator for any tables
o) complete charset collection for each valid character
o) database of mysql functions, data types
o) multi-threaded testing environment for any queries
o) query results comparer for any queries
o) rainbow, which will provide help in making good coverage of functions, etc.

What's needed still? Well, I would like 56 hours in a day ... More posts later when v0.001 of the integrated …

ok folks. here's some results:

mysql> select last_errno,count(*) from 
queryqueue group by last_errno;
+------------+----------+
| last_errno | count(*) |
+------------+----------+
|          0 |  1600796 |
|       1048 |     1971 |
|       1053 |        1 |
|       1139 |       35 |
|       1267 |    19722 |
|       1270 |     4243 |
|       1271 |     8944 |
|       1416 |     2284 |
|       1580 |    23225 |
|       2003 |       28 |
|       2013 |     1606 |
+------------+----------+
11 rows in set (0.00 sec)

error 2013 means lost connection to server (read: server crashed).
so there are many bugs found already. 1606 crashes out of 1.6 million
executed queries, is great.

check my rss feed for the exact bugs ...

I thought of a brilliant QA plan. Create sql rainbow tables! Just trust me, this system of QA will be very powerful in bug finding. Especially important will be the comparison of old/new versions of mysql. Writing a prototype now, on my day off.

Just during prototype development, I've discovered a handful of bugs. I'm testing all functions that are documented in the manual. This is alot that I've written up:

mysql> select count(*),category from func group by category;
+----------+--------------+
| count(*) | category     |
+----------+--------------+
|        6 | arithmetic   |
|        7 | bit          |
|       60 | casting      |
|       47 | comparison   |
|       52 | datetime     |
|       18 | encryption   |
|      127 | geometry     |
|       17 | information  |
|        7 | logical      |
|       32 | mathematical |
|        9 | misc         |
| …

I've been thinking about this today. My attempts thus far to write random queries which stress various parts of the server have worked, and served a purpose - but have been rather huge and clumsy.

For example a 5 union select, each comprising of 7 joins did have it's value.. The problem with huge random queries is simply that the results cannot be verified as easily as they were constructed (unless you run the same on various DBMS). These huge tests are mostly only good for checking if the server crashes.

Now, I wish to try write testcases that have these properties:

• small
• results are self-verified
• each test is random, but reproducible given a seed value
  

Tonight I'll throw together a code-generator to do simple insert/select and post any interesting results here later.