When we optimize clients' SQL queries I pretty often see a queries with SQL_CALC_FOUND_ROWS option used. Many people think, that it is faster to use this option than run two separate queries: one - to get a result set, another - to count total number of rows. In this post I'll try to check, is this true or not and when it is better to run two separate queries.

For my tests I've created following simple table:

PLAIN TEXT SQL:

1. CREATE TABLE `count_test` (
2.   `a` int(10) NOT NULL AUTO_INCREMENT,
3.   `b` int(10) NOT NULL,
4.   `c` int(10) NOT NULL,
5.   `d` varchar(32) NOT NULL,
6.   PRIMARY KEY  (`a`),
7.   KEY `bc` (`b`,`c`)
8. ) ENGINE=MyISAM

Test data has been created with following script (which creates 10M records):

PLAIN TEXT PHP:

1. …

In the Enterprise Team we had the need to track how many connections to the MySQL server in parallel. By "used" I mean running queries and not just idling around.

And the make it more complicated we needed the information not just every few seconds (while true; do echo "SHOW PROCESSLIST" | mysql; sleep 1;done) but when it happens.

Proxy to the rescue we now have a script which tracks the global state of all connections going through the proxy and dump them to stdout when their state changes:

2007-08-28 21:39:55
  #connections: 20, #active trx: 3, max(active trx): 4
  [792] (merlin@merlin) (COM_QUERY) "SET autocommit=0" (state=started)
  [794] (merlin@merlin) (COM_QUERY) "SET autocommit=1" (state=started)
  [795] (merlin@merlin) (COM_QUERY) "SET autocommit=1" (state=started)

• when the query comes in, we add it with its thread-id to the global table
• when the query is done, we …

I believe we wrote about this before, but this topic popups again and again.
Today I've read opinion that if we have clause WHERE has_something=1 we should have index on column `has_something` (the column has two values 0 and 1).

In reality the right answer is not so simple.

Let's look next table

PLAIN TEXT SQL:

1. CREATE TABLE `testr` (
2.   `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
3.   `name` varchar(32) NOT NULL,
4.   `has_something` tinyint(3) UNSIGNED NOT NULL,
5.   PRIMARY KEY  (`id`),
6.   KEY `has_something` (`has_something`)
7. ) ENGINE=MyISAM

with 20.000.000 records.

And in first case has_something=0 for 90% of rows (with random distribution)

PLAIN TEXT SQL:

1. mysql> SELECT cnt0/cnt FROM (SELECT count(*) cnt0 FROM …

Dana Blankenhorn over at sister publication ZDNet has a great analysis of the looming Microsoft Sharepoint threat. As Dana points out, it's a threat that open source is well-suited to meet and beat, but doing so will require some project coordination.

Interestingly, it's a threat that the proprietary Enterprise Content Management vendors have rejected as credible, even as Sharepoint boots them out of customer accounts. While ECM, operating system, database, etc. vendors sleep, Sharepoint is gaining ground.

If only open source could get its act together:

...

Where can you get access to some of the most informed people from MySQL and the community, for free?

The answer: at MySQL Camp. And then throw in lunch and breakfast for free, being able to influence the session topics and you have quite a package deal.

So it is strange why so few people took up the offer in New York this year!?

My talk was about the BLOB Streaming engine, MyBS, and I have posted the slides: Presentation - MySQL Camp 2007: The BLOB Streaming Project.

OK, so I got pretty much ragged about the name, MyBS. Why, I was asked, did I name it that? Jay, even suggested a session to find a new name for the engine! Thanks, Jay, very considerate of you... :)

But it was quite unnecessary, because I really can't see what the …

About year ago Peter wrote about redundant indexes and mentioned sometimes it is good to leave two indexes, even one is first part of another. I'm speaking about BTREE indexes, for example, KEY (A), and KEY (A,B). From SQL point of view KEY(A) is not needed, as for queries like WHERE A=5 the index (A,B) also can be used.

But there is case when for performance it would be good to have both

Let we have the table

PLAIN TEXT SQL:

1. CREATE TABLE `userinfo` (
2.   `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
3.   `name` varchar(64) NOT NULL DEFAULT '',
4.   `email` varchar(64) NOT NULL DEFAULT '',
5.   `password` varchar(64) NOT NULL DEFAULT '',
6.   `dob` date DEFAULT NULL,
7.   `address` …

ActiveGrid, Inc., a leading provider of Enterprise Web 2.0 solutions, has announced it is working with MySQL to simplify the migration of legacy applications to modern web architectures. Last week, the companies published the "Microsoft Access to MySQL Migration Guide," a quick guide to migrating MS Access applications to a CIO Safe, Enterprise Web 2.0 deployment architecture. Next month, the two companies will be offering a free webinar covering Application Migration to Enterprise Web 2.0.

When I started, I found the section on backups in the manual a bit confusing. I wanted to know, what’s available for InnoDB backups that would correspond to “exports” and “hot backups” for Oracle. I had trouble extracting a clear answer from the manual.

Here’s what I’ve found over time, to the best of my knowledge.

Assuming you don’t want to buy extra software (eg, Zmanda), you can use mysqldump (”export”) and/or LVM (”hot backup”) for your backup needs. (mysqlhotcopy is not for InnoDB.) Replication is nice, too. Here’s a bit about each method.

1. mysqldump (think “export”)
You can get a full backup, even for point-in-time recovery, using mysqldump. Mysqldump is sort of like Oracle’s export. Dumps the database, and you can use it to import it back. It creates a bunch of text commands that you can redirect into a mysql client to do the import. And mysql has a tool for …

In the Enterprise Team we had the need to track how many connections to the MySQL server in parallel. By "used" I mean running queries and not just idling around.

And to make it more complicated we needed the information not just every few seconds (while true; do echo "SHOW PROCESSLIST" | mysql; sleep 1;done) but when it happens.

Proxy to the rescue we now have a script which tracks the global state of all connections going through the proxy and dump them to stdout when their state changes:

2007-08-28 21:39:55
  #connections: 20, #active trx: 3, max(active trx): 4
  [792] (merlin@merlin) (COM_QUERY) "SET autocommit=0" (state=started)
  [794] (merlin@merlin) (COM_QUERY) "SET autocommit=1" (state=started)
  [795] (merlin@merlin) (COM_QUERY) "SET autocommit=1" (state=started)

• when the query comes in, we add it with its thread-id to the global table
• when the query is done, we track …

On Friday I committed code to 3.0 trunk to allow people to execute an MDX query on a Mondrian server and get the result back in a tabular format. This particular code to “flatten” an OLAP cube into rows was written by Julian Hyde, the lead developer and founder of Mondrian OLAP a.k.a. Pentaho analyses.

If you run the Pentaho demo on your box and then look at the Analyses sample, you could see something like this:

Suppose you wanted to get this exact data to work with, create analytical data, exports, … Well, now you have the option of doing it in Kettle:

What you do is create a database connection to the database that Mondrian reads from, hand it the location of the Mondrian schema (catalog). You then click on the MDX button in JPivot and copy the MDX query to the “Mondrian Input” step in Kettle. That’s all it takes.

You then can preview …