We have an application which stores massive amount of urls. To save on indexes instead of using URL we index CRC32 of the URL which allows to find matching urls quickly. There is a bit of chance there would be some false positives but these are filtered out after reading the data so it works all pretty well.

If we just process urls one by one it works great:

PLAIN TEXT SQL:

1. mysql> EXPLAIN SELECT url FROM 124pages.124pages WHERE url_crc=484036220 AND url="http://www.dell.com/";
2. +----+-------------+----------+------+---------------+---------+---------+-------+------+-------------+
3. | id | select_type | TABLE    | type | possible_keys | KEY     | key_len | ref   | rows | Extra       |
4. +----+-------------+----------+------+---------------+---------+---------+-------+------+-------------+
5. |  1 | …

Yesterday I went through hell while testing the MySQL Proxy partition Lua scripts I am working on in a high concurrency environment. I am sending multiple queries to the server and build up a combined result set in read_query_result - something like this. The proxy returned weird results complaining about multiple result sets being sent from time to time at totally different places even though I was sure that for each query only one result set has been sent. And some of the results were just wrong, a lot of our tests failed unexpectedly.

After long hours flicking almost every switch I simply removed the Lua module declaration from the main LUA script passed to the proxy via proxy-lua-script since it was the only thing left that …

We have now completed the RC3 build and will test it over the weekend. If we do not find any major issues we will release on Monday. There are a bunch of fixes in this release again. Here are the higlights.

• Syntax Highlighting activated for all schema object editors
  The syntax highlighter we used for the Lua scripts has proven itself since the last release so we decided to replace the dull standard edit controls
• Canvas Speed Optimizations
  Alfredo found a stupid bug that was disabling all the caching that he built into the canvas. The speed has improved a lot and there is still room for a lot more
• Description Palette
  We have now enabled the Description Palette that directly shows the description text associated with the currently selected schema objects. This makes instant documentation of schema objects on the diagram as well as the overview page much simpler
…

On Friday, after weeks of benchmarking T5120 and T5220 and studying the Sparc T2 (Niagara 2 chip) architecture, I finally put one in production and the results have been very promising. Though Friday evening wasn't a peak period, we experienced 62% more throughput than the previously deployed V210. I expect T5120 to be able to handle our peaks without breaking a sweat.

We'll have to take a hit in certain database operations to benefit from this 62% gain. (Update: however, luckily, those operations do not occur everyday.) I will be presenting results of my benchmarks and information at the MySQL conference. If you are evaluating Sun servers for MySQL, you will find my session very interesting.
…

Recall that I’ve claimed that it takes 28 years to fill a disk with random insertions, given a set of reasonable assumptions.  Recall what they are:

We are focusing on the storage engine (a la MySQL) level, and we are looking at a database on a single disk—the one we are using for illustration is the 1TB Hitachi Deskstar 7K1000.  It has a disk seek time 14ms and transfer rate of around 69MB/s. [See tomshardware.com] We insert random pairs, each 8 bytes.  So that’s 62.5 billion pairs to fill the disk, and at 4KB-size blocks, that 2^28 leaves (= 2^40 bytes / 2^12 bytes/leaf).

Now, my analysis requires each insertion to induce a disk seek.  Suppose we do something clever with main memory.  After all, we have this main memory hanging around.  It should be possible to buffer up some insertions, and once we fill up main memory, insert key/value pairs that belong on the same leaf.  Thus, fetching a …

I recently was interviewed by Packt Publishing for their Impackt '08 web pages:

Ever since the formal adoption of the term in 1998, Open Source has experienced growth and adoption rates that defy pressures and suggestions that it’s a viable option for enthusiasts and geeks only. Governments, corporations as well as small businesses have begun to choose Open Source over proprietary software. However, with the global economy facing an uncertain future, how will open source be impacted? Can it continue to grow despite this?

With these questions in mind and more, Packt approached some people at the heart of this movement to understand their take on the future of open source.

The …

Yesterday I chatted with Jay Pipes at his home in the Buckeye State.  Jay is the North American community manager for MySQL and very recent Sun employee.  We chatted about whats up in the world of MySQL and all about the MySQL Conference thats coming up in 10 days (April 14-16, in Santa Clara).

My interview with Jay (10:01)  Listen (Mp3)   Listen (ogg)

 
Photo by Sebastian Bergmann

Welcome to the 91st edition of Log Buffer, the weekly review of database blogs.

For a change, let’s begin with some PostgreSQL stuff. On Tending the Garden, Selena Deckelmann gives her retrospective thanks to those who attended and presented the PostgreSQL Conference East.

On Esoteric Curio, Theo Schlossnagle gives his thoughts on the keynote address by Joshua Drake, touching on the perennial versus, Postgres vs. MySQL.

Hey, there was a MySQL …

A bit late in the game, but maybe somebody would be interested in working on this proposed project of mine:

PlanetMySQL currently is merely an aggregator of submitted RSS feeds, with some functionality for filtering content to keep the discussion on topic. Due to its high volume of posts, many gems get "lost in the noise" and are hard to retrieve.

We'd like to expand the functionality of PlanetMySQL significantly to provide more possibilities for community participation and interaction. For this project, we are looking for a talented PHP hacker to set up a site that provides the current functionality and more:

• Voting on articles/blog postings: it must be possible for logged in users to cast a vote on articles, similar to the …

For the past few months, like Baron, Jeremy and Keith, I have been consulting KickFire (formerly known as C2App). There is another startup currently in stealth mode with some very impressive solutions for MySQL. Unlike Kickfire, this other startup isn't SSD based. I was hoping they will be ready for announcement at the conference as well, but it seems they will need some more time. I cannot go into much detail on this startup at this point.

I have been wanting to write on KickFire but I certainly won't be able to beat Baron. He does a wonderful job in capturing what is …