This week throngs of MySQL developers, users, and enthusiasts descended on silicon valley. Apparently the valley’s cooling system can’t keep up because as they arrived the outside temperature went up into the 90s (32s for those of you who choose to use a sane temperature measurement system). I’m not attending the conference this year but I almost wish I was to get some of the air conditioning. It’s supposed to cool off and rain on Saturday. As the brain power leaves I suspect the valley is going to cool and moisture in the air will condense into rain, or tears.

I bet you thought this post was going to be about Oracle and Sun, sorry. I think the weather is more interesting.

I was reading through the manual and noticed that myisamchk parallel recover option is still listed as beta code. The feature was added in 4.0.2 which was released in july 2002. This means it’s been in beta longer than gmail

While updating the mirror last week I was surprised to see that the newest MRU MySQL release is numbered 5.0.79. Previously enterprise releases had even numbers and community releases had odd numbers. I posted the question in #mysql-dev and HarrisonF was kind enough to explain it all.

MySQL 5.0 is running out of version numbers. There are limitations in mysql_get_version(), the executable comment syntax, and other places that mean MySQL can only have two digit release version numbers. MySQL Enterprise has started using odd and even version numbers to extend the life of 5.0.

This raises a few questions. What will happen to 5.0 when it runs out of release numbers? Is community going to be sacrificed to give enterprise more versions to use? Are the version restrictions going to be fixed in the future? For example if a feature is implemented in a community

  [Read more...]

The #mysql-dev irc channel on freenode was created with the idea of getting the community people more involved in active discussion about mysql internals and development. When the channel was first created this happened for a few weeks and I was pretty happy to be able to observe and participate in the discussion. Now it’s mostly idle.

It seems that some people at Sun think there is still active discussion or internal developers paying attention to the channel because the GSoC web page directs people to #mysql-dev as a point of contact. The problem is there isn’t anyone there answering questions. I’ve seen quite a few people over the past few weeks ask questions that have gone unanswered. I think it’s time to restart the movement to open development and using #mysql-dev for discussion.

So it turns out that the answer to the question "when computers and television converge, what will we have" is "computers". link.

So the next question is "when computers and telephones converge, what will we have?"

I think the answer is going to also going to be "computers". Just very portable ones.

A few months ago I got a strange email from one of my clients that contained two very simple looking select queries. The only difference between the two queries is that one included the distinct keyword and the other didn’t. The strange part is that the query that used distinct returned zero rows. I spent a few days narrowing down the clients data into a small test case then created a generic test case from that. I also traced the problem to the code that decides which index to use for a group by loose index scan which can be used to resolve queries using distinct.

The example can be found in the bug and in this sql file. My patch was a step in the right direction but not complete enough to solve all the issues. Since this isn’t a crashing but I was tempted to

  [Read more...]

I wish I could say

	printf("%*.16xs", n, p);

where n is a size_t and p is a void*, and it would output n bytes as hex.


And I want to be able to say

	printf("%*qs", n, p);

and it would output n bytes as a C-style backslash quoted string.


Yes, I can write functions that do all this, but I have to worry about buffers, allocate buffer space, and so forth. Plus its a lot of noise and clutter, for just outputing potentially unprintable strings, often just for logging and debugging.

A feature of the MySQL server that is used a lot, and yet is a source of much user confusion, code complexity, and multiprocessor lock contention, is logging. Query logging, slow query logging, and the new 5.1 feature, "log to table".

I've removed most all of that stuff from Drizzle (and removed two or three sets of now-no-longer-necessary mutex locks in the process), and replaced it with hooks into a logging plugin subsystem, and have implemented two plugins for it. One logs to a file, and the other logs to syslog.

The output looks almost completely unlike the current MySQL logging. There are no hash-prefixed pseudocomments, for one thing. And there is no distinction between the query log and the slow query log. Queries get logged, and the amount of time each query takes gets logged with it. This subsumes the "micro-slow patch" that is spreading around in the MySQL legacy



  [Read more...]

Everyone and their mother are throwing out their predictions for 2009 nowadays, it’s a new fad. It’s like you’re not cool anymore if you don’t have twitter, a Mac, and a set of random predictions for the next 12 joyous months.

So I decided to throw in a few ideas of my own to be part of the cool crowd again (how much cooler can I be already, you might think, and I wouldn’t blame you).

 

Disclaimer (read it, tough guy)

What this post is:

• about the future of technology and the Internet, 2009 and beyond.
• my ideas on what is going to happen or should happen. If they happen to match someone else’s ideas – it doesn’t mean I ripped them

  [Read more...]

I've been involved with the Drizzle project since very soon after it began, working on it on nights and weekends.

That has just changed. As of today, I'm no longer a MySQL Professional Services consultant, instead I'm part of a new division of Sun

Much of my time is to be spent working on Drizzle, with a focus on plugin interfaces and making it work well in Extremely Large distributed environments.

I will be blogging heavily about what I am doing. How I sort that blogging out between my personal LiveJournal, my (mostly unused) Sun employee blog, and maybe some other blog system, remains TBD.

This is going to be fun.

A friend suggested the creation of a "Technical Professional Purity Test", and suggested two questions:

Have you ever have you ever implemented a solution you knew would doom the client?

Have you ever yelled at your manager for interrupting your workflow?


Please, suggest more!

Over in CNET, Matt Asay has posted an article The open-source job shortage, talking about large enterprises' need for developers with deep MySQL experience.

While he is correct about the need for talent with that skillset, there are plenty of effective solutions.

A number of months ago, Harper Reed asked me where he could hire MySQL talent, and I told him to take his existing staff, and run them thru MySQL training (http://www.mysql.com/training/). That seems to have worked for him. That's now my stock answer when people ask where they can hire MySQL talent.

When you need to go up to the next level, get and read the book High Performance MySQL, Second Edition. The book is basically several of the very best MySQL





  [Read more...]

Dell has obtained a trademark on the term "Cloud Computing".

http://tarr.uspto.gov/servlet/tarr?regser=serial&entry=77139082

People have been talking about computing in the cloud for years now, and network designers have been using a cloud icon to indicate "services on the internet out there somewhere" for over a decade.

This trademark needs to be killed, it born generic.

Why may LIMIT only take a fixed number? Not even a user variable, let alone an expression.

Wouldn't it be useful to set a user session variable that always gets applied as an implicit limit value?

Both of these things could be done in Drizzle.

Last week I was working with a client that rediscovered a bug where setting expire_logs_days and issuing a flush logs causes the server to crash. It’s MySQL Bug #17733 if you want to have a look. Seeing MySQL crash was enough inspiration to fix something that I and others have wanted to fix in MySQL for years.

Currently a flush logs command tries to flush all of the following logs in order:

• General Log
• Slow Query Log
• Binary Log
• Relay Log
• Store Engine Logs (If available)
• Error Log

The reason I wanted to fix this is because my client was issuing a flush logs to rotate the error log on a server with no replication. The crash was caused by replication. With individual flush logs it’s less likely for this to happen again in the future.

  [Read more...]

Several months ago I was invited to a special FOO camp at the O'Reilly Campus, about social networking and the social graph. One of the things that the ora people did was record an interview video of each attendee. They've now put all those videos up on youtube.

Here is mine.

Have you ever executed a query from the MySQL command line client only to find that the output wrapped and the result is unreadable? In the past you have to run the query again with \G instead of ; or \g to get it to display the output in a vertical mode. My feature in MySQL 6.0.4 fixes that. The auto-vertical-output option tells the command line client to display the results in vertical format if the results are going to be too wide to display horizontally. It does this without re-executing the query because MySQL passes the length of each column in the result set. If the client isn’t able to determine the width of the screen it will default to 80 chars.

This is a continuation of the MySQL User Conference replication notes part one.

The session is opening up talking about failover. The shared disk in this case is drbd. DRBD is a fine product for replicating block devices of single disk systems. It’s made redundant by raid and doesn’t provide as much protection as binary log failover. You can find my notes on why I don’t recomment DRBD for MySQL in drbd in the real world.

Lars went a bit quick through the other two configurations. I’ll try to review the slides and post comments.

The next configuration is using federated. The federated storage engine has many problems that make it almost useless for any production deployment. Mats says, “Federated isn’t the fastest engine in the world”.

  [Read more...]

I’m attempting to live blog corrections and notes while sitting in the replication tutorial. Lars is covering available options in MySQL replication. I’m going to attempt to cover some recommended best practices and things that are possible to do in MySQL but should be avoided. Please keep in mind that I’m writing this during the presentation. If anything is confusing post a comment and I will clean it up.

When designing a MySQL architecture that are several possible configurations. Two that should be avoided are dual master where you write to both masters. Configuring replication in a dual master dual writer setup means there is no single authority on the data. There is also no need to write to both masters as this doesn’t give you any performance improvement. Each master has to process the same sql statements. One step further is circular

  [Read more...]

I woke up this morning to a nice surprise. Peter Zaitsev from MySQL Performance Blog mentioned my fix for a crash in MySQL. Thanks Peter! That reminded me that I need to write this post about another fix.

A storage engine is called a “handler” in MySQL internals. Handler events are per row at the storage engine layer. For example an insert query that inserts 3 rows will cause one Com_insert increment and 3 handler_write increments. The way these statistics are implemented the storage engine is responsible for incrementing statistics itself. Innodb is missing function calls to increment the handler_delete and handler_update status variables.

  [Read more...]

While working with a client we found a query that can crash MySQL. It has to do with ref_or_null outer joins where the table contains a null value. MySQL fails to tell the storage engine to initialize the index before reading from it. It crashes when used with MyISAM tables but it could result in an incorrect key file for table error with other storage engines. You can find the test case and patch in bug #34945. I’ve waited to blog about it until MySQL approved my patch. As of this writing my patch has been approved and committed for release in 5.0.60.

My employer, both MySQL and Sun, will reimburse me for my internet connection.

I wonder if I could just buy FON routers for my favorite cafes, and then expense them? It would let me have better and more reliable internet access then I get thru the crappy little wireless routers from Quest and Comcast...

[Updated: 2008-03-05 Jeremy writes to tell that I left out the provenscaling.com contact form submission on the 21st and the blank email with a pdf on the 27th]
[Updated: 2008-03-26 Rob Young from MySQL AB sent me an email on the 17th saying, "We are currently working to implement PayPal to cut down on the email interactions we have in place around our manual fulfillment process. Testing is going well and we should have this implemented in the next few weeks."]

Recently Proven Scaling decided to purchase a MySQL Enterprise subscription. We will be evaluating Enterprise to see if it’s knowledge base is valuable to our customers as a supplement to Proven Scaling’s consulting services.

The subscription process seems remarkably manual on MySQL’s part. It involves them sending us five emails (not counting the mysql.com registration), all

  [Read more...]

In Issue 3 of MySQL Magazine Peter Zaitsev writes, ” I would set [innodb_log_file_size] to 256 megabytes for small sized boxes up to 5% of the total memory on the big boxes.” I tend to set my log file size to around 128M. Depending on your system setting the log file size around a few hundred megs should be the right balance between preventing checkpointing and still having fast recovery times.

The manual states, “Sensible values range from 1MB to 1/N-th of the size of the buffer pool, where N is the number of log files in the group.” The sample my.cnf files that come with MySQL state to set the innodb_log_file_size to 25% of the innodb buffer pool size. I think this mistakenly advises people to set their log files much larger than they need to be causing unnecessarily long recovery times.

Crash recovery time is an important feature of InnoDB. In

  [Read more...]

When I received and email almost two years ago about a announcement that MySQL was going to release and enterprise product I was very excited. I was looking forward to a redhat style model of vetting releases in the community then offering a proven stable version to paying customers. I saw it as a great way to for MySQL to generate revenue as well as eliminate the need for people to stay a few releases back from the head and guess when to upgrade.

The release shocked me. What was originally emailed to me and the final plan were two very different things. It was a plan to hand paying customers bleeding edge code that had been tested only by MySQL’s QA team. It seems MySQL has forgotten the years of testing by millions of community members that has given MySQL the stability we have grown to trust. I predicted the instability of

  [Read more...]

MySQL Bug Tracker

I realize that MySQL is in a transitional phase making it’s development more transparent to the community so this entry isn’t really a rant but something that will hopefully ease the transition. The public MySQL bug tracker doesn’t seem to have any way to view the history of status changes to a bug. Things like changing a bug from open to inactive. I assume there will always be things that should be hidden from bugs like links to sites internal to MySQL AB but things like simple status changes should be visible to the public.

For example on this Bug #20358 Heikki posted a comment that he was moving the bug from open to unable to repeat. Did he actually make the change? I have no idea because I can’t access the status log. Was it ever changed from unable to repeat back

  [Read more...]

In response to brad's post and daveman692's post, about Thoughts on the Social Graph.

I have posted the following:

I came home from FOOcamp with my mind buzzing with something similar. It was in the zeitgeist, I guess.

Between being annoyed at some of 6As recent actions, and annoyed that if I left LJ, I would lose a lot of valuable social network information. And then the sessions

  [Read more...]

Dear Lazyweb,

I have all my old .netscape and .mozilla directories for all my past installations of Netscape, Mozilla, and Firebox, going back to 1996. I've got cert.db, cert5.db, and cert7.db files.

I want to extract out all of the certs and keys that I had added to those databases, and then import them into my current ~/.mozilla database.

Reading the docs and futzing around with NSS signtool, certutil, and pk12util isn't getting me very far.

Surely there is a tool or technique do doing this...

Recently I’ve been making more modifications to the MySQL source. Part of making modifications is testing them. For years I’ve been curious why there are both tests and mysql-test sub directories in the source tarballs. Arjen Lentz tells me that tests is old and that mysql-test is the current testing framework. This makes sense since the manual testing pages are all about mysql-test.

Are there plans to clean up the remnants of the old testing framework? What else is there in the tarball that’s outdated and needs to be removed?

I worked out these rules from speaking at MySQLcon, and attending OScon this year.

• Speaking at geek conferences is fun.
• Don't put your talk on your slides!
• Don't put everything on the slides, don't put the outline on your slides, don't put a lot of stuff on the slides.
• Put questions on the slides, instead. Each question is the one you answer while that slide is up.
• You can occasionally have an answer, a good quote, or a graphic.
• If someone can't QUICKLY write down what is on the slide, it's too complex.
• Do write speaker notes. Make them as complete as possible. Try to memorize them, but don't just recite them. You can refer to them if you have to during your talk.
• Practice your talk at least once. Check time. You may feel silly talking to yourself in front of the wall, but you will feel dumb instead if you run

  [Read more...]