I was thinking today of how people tend to use MySQL in modern applications and it stroke me in many cases MySQL is not used to process the data, at least not on the large scale - instead it is used for data storage and light duty data retrieval. Even in this case however the cache (ie memcache) is often implemented to offload MySQL.

When it comes to large scale data processing other techniques are used. Google uses some bigtable based technologies for search and Google Analytics (which is quite typical log processing application), when you look at recommendation task as well as a lot of other personalized services they may not use MySQL (and SQL at all). In many cases you would see absolutely custom solutions are implemented, in other cases you would see Hadoop or Sphinx used to crunch data in parallel or it even could be …

So I read an interesting blog recently that's linked from Sheeri Kritzer's Blog. It went to say that "Embedded SQL" is bad. Which is to say that stored procedures are good and that you should never let a developer mess around with the database. In Sheeri's blog there is an ongoing argument about it as you read this post.
Well, I already wrote in my blog that I do support the stored procedures approach. Maybe not fanatically as the original blog post, but I do agree that developers are not really interested in the database, just the results they are looking for.
So I had an idea. I was thinking how can help manage the way developers interact with the database, to give both …

Maatkit (formerly MySQL Toolkit) has for some time been released both as a bundle, and as individual tools. It’s too much work to maintain the individual packages, and I don’t think it really benefits anyone much, if at all. While the tools will still be versioned separately, I’m going to discontinue releases of the individual packages, and just release the one uber-package from now on. This will also make it easier for me to manage the name change, but that’s just an extra incentive; I’ve been considering this for a while.

I am so lucky I married an archaeologist. Choosing a new name for MySQL Toolkit has been a hassle. I wanted to avoid a literal name, such as, um, MySQL Toolkit. Short is good. And so on, and so on. All the while, the Phoenix/Firebird/Firefox naming debacle was in my thoughts. I only want to do this once. At first I tried not to stray too much from the current name.

Premature Optimization ... we all know it. The proxy is now in a state were we can start to optimize the code a bit. After getting some complaints about the performance with keepalive I took a deeper look into the problem and came up with:

• a global script-cache with reload on cache
• less GC-runs
• less overhead per lua-script-call

It is all in SVN now, enjoy it.

To test if we make progress I use mysqlslap in its most basic form:

$ mysqlslap --host=127.0.0.1 --port=4040 \
  --create-schema=mysqlslap --iterations=1000 --concurrency=10

which only connects and closes the connection again.

The run-time over 10.000 connects is

proxy, no cache, lua_gc()  11.553s
proxy, cache, lua_gc()      7.065s
proxy, cache, no lua_gc()   6.458s

I just implemented a script-cache for MySQL Proxy and had to learn a bit more about lua internals. As a result I wrote two functions which are generic enough for general consumption:

• luaL_loadfile_factory()
• luaL_loadstring_factory()

They also show how to write a lua_Reader for lua_load().

The #lua channel on freenode was a great help, as always.

LUA provide several loaders:

• luaL_loadstring
• luaL_loadbuffer
• luaL_loadfile

and you can create your own if you have to.

Now, why the need for a new loader ?

The proxy we have

• a global scope caches the scripts
• this cache reloads the scripts if they changed on disk
• each thread executes the same script
• each thread has its own …

Damien Katz and the CouchDB development team are proud to announce:

CouchDB version 0.7.0 is now available.

This release is a major milestone in the project’s history.

Key features include:

• a REST API using JSON instead of XML for data transport,
• a JavaScript view engine based on Mozilla Spidermonkey,
• a GNU Autotools build system supporting most POSIX systems (Noah Slater),
• a built-in …

So, I got my new MacBook 13" - since Apple updated the chipset. That ups the speed to 2.2GHz (it's a Core 2 Duo), has better graphics (apparently, don't think I care too much for that), but most importantly, it can now handle up to 4GB of RAM. Not that you should get the 2x2GB from Apple directly (the other ram upgrades are not that expensive with them actually) but here they decided to ask for a whopping $1250 or so. Daft. Got the memory sticks from elsewhere, of course. The 4GB is glorious! No swapping, very responsive, even with lots of stuff open. Great.
Note that I ordered the MacBook directly from the Apple store, because the local shops in AU didn't have the new model yet (different warehouse apparently).

New Macs come with OS X 10.5 (Leopard). Very shiny. Time Machine is particularly cool, as is the GUI profiler that's part of Xcode. Lots of other little things improved and fixed. Although I'm not sure I like the new …

Welcome to the 71st edition of Log Buffer, the weekly review of database blogs. There were so many blogs covering this week’s Oracle Open World that I could have devoted this entire edition only to that. But that would not be fair to those other DBMSs, all of which are fine DBMSs in their own [...]

Background Knowledge

The MySQL database v4.0.23 is using the default character set of “Latin1″. When the database was created I had no knowledge of character sets other wise it would have been “UTF-8″.
The web pages are using a character set of “UTF-8″.

Problem

Data being queried from a MySQL database that contains French accent characters will not render correctly in the browser even after applying PHP htmlentities().

Example code: $string = htmlentities($string , ENT_QUOTES, “UTF-8″);

Solution

The queried data from the database was inputted using the character set “ISO-8859-1″. I …