Lua is a great programming language and I use it as embedded scripting language in

• MySQL Proxy
• wormhole Storage Engine
• lighttpd

What impresses me most it's flexibility. What ever I come up with is only a few lines away ... and I have crazy ideas. Like profiling lua scripts with KCacheGrind.

For profiling lua-scripts you can use

• luaprofiler
• …

Hartmut was asking me some time ago how table discovery in the storage engine interface works. After reading through the code from ndb, archive and memcache I was a bit disappointed: all of them are just copying the definition in binary form around.

For the wormhole SE the lua-file has to create table structure on the fly. You only drop in the .lua into the db-folder and a SELECT will pick it up automaticly.

We use a small function which returns the table definition:

function discover()
        return {
                { name = "fld1", type = 1 },
                { name = "fld2", type = 2 },
                { name = "fld3", type = 15, length = 64 }
        }
end

A SHOW CREATE TABLE against the table shows us:

root@127.0.0.1:test> show create table foobar\G
*************************** 1. row ***************************
       Table: foobar
Create Table: CREATE TABLE `foobar` (
  `fld1` tinyint(4) …

I got a few comments about my last example not describing a wormhole, but a whitehole. Time to improve the picture a bit and getting data from another dimension on a shorter route than the long standard way.

To use the picture let's take a look at what has been done:

root@127.0.0.1:test> select * from finance where symbol = "MSFT";
+--------+------------+---------------------+--------------+------------+-----------+-----------+----------+
| symbol | last_trade | trade_time          | trade_change | open_trade | max_trade | min_trade | volume   |
+--------+------------+---------------------+--------------+------------+-----------+-----------+----------+
| MSFT   |      29.84 | 2007-10-05 00:00:00 |         0.13 |      29.84 |     29.99 |     29.73 | 45016520 | 
+--------+------------+---------------------+--------------+------------+-----------+-----------+----------+

root@127.0.0.1:test> select * from finance; …

MySQL has a quite unique feature: the pluggable storage engine interface. Thanks to it MySQL supports different Storage Engines for different needs: MyISAM is perfect for heavy read, InnoDB for transational data and blackhole .... for sending data to /dev/null.

Thanks to some advance science we now have a wormhole Storage Engine. While the blackhole can only be written to but nothing ever comes back, the wormhole is the inverse of it. You get data from another galaxy, but all writes might have no effect on your side. Sounds useful ?

What's the deal ? The wormhole SE is a lua-based storage engine. The data is "stored" in a script-language. ... Ok, this explaination doesn't help very much. Let's take a look at an example:

The CREATE TABLE for a wormhole table is pretty simple. No magic:

1
2
3
4

When seeing that the MySQL Proxy was released, I decided to try to experiment with it since I see strong potential with this tool, both for replication and for other uses (recall that I'm a replication guy, so this is my primary focus). I'm actually on vacation, but this will of course not stop me from tinkering with things (I know, I'm just a hopeless case in this aspect ;) ).

After reporting a minor bug, I managed to build and run it with some sample scripts. I'm using Kubuntu Feisty, and had some initial problems, but it was actually pretty straightforward. I'll repeat the steps anyway, in case anybody else have problems.

1. Get the source from the repository svn co http://svn.mysql.com/svnpublic/mysql-proxy/ mysql-proxy
2. Make sure you have all packages necessary. Several of the packages …

lua + FastCGI

I was looking for a FastCGI backend for lua to have a asynchronous brother for mod-magnet in lighttpd.

Same as mod-magnet this embedding of lua is not meant to replace Frameworks like Rails or Spring, nor do I want to write average PHP application in it. I use this magnet to write small scripts (for this project it was 140 lines of lua) which is going to be executed at least 500 times a second.

That is a range where the setup-cost for a request matters. I don't want to load the session, nor do I want cleanup the whole environment for each request. I need a way to store connections to the database over multiple requests and I have to cache content from the database in the application.

I needed: - a byte-code cache …

lua + FastCGI

I was looking for a FastCGI backend for lua to have a asynchronous brother for mod-magnet in lighttpd.

Same as mod-magnet this embedding of lua is not meant to replace Frameworks like Rails or Spring, nor do I want to write average PHP application in it. I use this magnet to write small scripts (for this project it was 140 lines of lua) which is going to be executed at least 500 times a second.

That is a range where the setup-cost for a request matters. I don't want to load the session, nor do I want cleanup the whole environment for each request. I need a way to store connections to the database over multiple requests and I have to cache content from the database in the application.

I needed: - a byte-code cache - …