I'm sure as a system administrator on any flavour of UNIX you appreciate the power of the few magic characters, such as #!/bin/sh, found at the first line of many scripts, commonly called a shebang. They allow you to treat a script almost as a binary executable. To run it, you just type its name at the prompt and don't have to worry if it's written in Perl, Bash, awk, Python or anything else. Just for your MySQL scripts you still type the verbose mysql < your_script.sql all the time, don't you?

I found this to become especially annoying (yes, programmers and administrators are lazy folks) now that every MySQL release gives me more and more powerful features: Where I before had to write a script in another language anyway, now I often find that MySQL allows me to do everything in plain SQL, even some complex data extraction tasks (using …

Hi All -

Just a quick note that I’ve written a follow up article on partitioning that’s in the MySQL Dev Zone article section for you to check out.  It shows how well defined segments of data can be easily managed and removed with partitioning.  Check it out if you get time and let me know how you’re using partitioning in MySQL 5.1 now!

Thanks,

–Robin 

DaveM talks about Ingo’s new SMP lock validator for linux kernel

A note reminding me to go take a look and see what can be ripped out and placed into various bits of MySQL and NDB. Ideally, of course, it could be turned into a LD_PRELOAD for pthread mutexes.

Anybody who wants to look deeper into it before I wake up again is welcome to (and tell me what they find)

In what I laughingly call “spare time” I started hacking on ha_file.cc, otherwise known as the FILE storage engine. My idea is relatively simple, I want to be able to store and access my photos from MySQL. I also want the storage to be relatively efficient and have the raw image files on disk, not tied up too much in any different format (my file system is pretty good at storing multi-megabyte files thank you very much) - it also doesn’t require any fancy things to re-use space when I delete things. I should also be able to (efficiently) directly serve the images out of a web server (satisfying the efficiency itch). You could also use something like DMF to migrate old rows off to tape.
So, I started some hacking and some designing and have a working design and a nearly basically working read/write implementation. I’ll share the code when it does, in fact, actually work (by “work” i mean reads and writes basic rows).
I’ve …

At the MySQL User's Conference in April, there were a number of new startups coming to market. One market that I think could be huge is for open source EAI (Enterprise Application Integration) software.  I worked in the EAI space for a number of years, helping take Active Software public in 1999 as VP of Marketing and then staying on with webMethods when they acquired Active.  The EAI market had huge growth during that time, fueled by the need for new internet systems to integrate with legacy applications.  The need still exists today, but the growth has leveled off as traditional EAI solutions have been too complex and expensive for most CIOs. 

I think there's a great opportunity for someone to come in and simplify EAI with a solution that works for 90% of the needs out of the box, with the ability for the most demanding customers to customize it to their own needs.  This is exactly the type of middleware that could …

My blog link at PlanetMySQL.org is still not updated so here is list of posts me and Vadim are written during last couple of weeks.

Group Commit and XA Some more benchmark results for MySQL 5.0 performance regression because of broken group commit.

Speedup your LAMP stack with LigHTTPD The article is about Optimization of Web Server part in LAMP stack in general and how you can use lighttpd to speed things up in particular.

How Web Services world affect LAMP Stack Article discusses some things which become different with Web Services application. Kind of followup to …

There are many questions how InnoDB allocates memory. I'll try to give some explanation about the memory allocation at startup.
Some important constants:
NBLOCKS=count of block in innodb_buffer_pool = innodb_buffer_pool_size / 16384
OS_THREADS= if (innodb_buffer_pool_size >= 1000Mb) = 50000
else if (innodb_buffer_pool_size >= 8Mb) = 10000
else = 1000 (it's true for *nixes, for Windows there is a bit another calculation for OS_THREADS)
So InnoDB uses:

• innodb_buffer_pool
• innodb_additional_mem_pool_size
• innodb_log_buffer_size
• adaptive index hash, size= innodb_buffer_pool / 64
• system dictionary hash, size = 6 * innodb_buffer_pool_size / 512
• memory for sync_array, which is used for syncronization primitives, size = OS_THREADS * 152
• memory for os_events, which are also …

Everybody knows that valgrind is great.

Well, I was observing a problem in some MySQL code, it looked like we were writing over some memory that we weren’t meant to be (as the structure hadn’t been initialised yet). But, seeing as this was memory that had been allocated off a MEM_ROOT (one of our memory allocators), valgrind wasn’t gonig to spit out anything.

This is because this bit of memory had already been allocated and subsequently “freed”, but then reallocated. The “free”ing overwrites the memory with garbage (which is what the MEM_ROOT code does) so that you should see crashes (and a pattern) when you do something bad.

The traditional way to troubleshoot this in to modify your memory allocator so that it just calls malloc() and free() all the time (and valgrind will trap them). We have some code in there to do that too. However, this requires …

My good friend Zak Greant spotted a news item at Techworld, where the European Commission appears to have changed its position on software patents.

Have a read, it's interesting and I hope that it really does indicate a new direction from the commission and not just one loose statement. If it is for real, that's absolutely great, but let's not be complacent and think "that's it". The pro-swpat lobbyists certainly won't give up, they have strong interests and stacks of cash driving them.
Of course it's not helpful to preach to the converted, but encouragement to continue on this path seems like a good idea. I'll happily give politicians kudos and encouragement when they do good stuff!

Also, if you haven't yet …

So what do I do on Memorial Day Weekend?

Well in theory I did not work.

Friday I went to go to see a SIFF, Seattle Internal Film Festival, movie in the evening. The fact that I can not remember what it was, and I think I gave it three stars means that I will not be recommending it to anyone. I have eight movies to see in three weeks (of which in the three weeks I will go to Boston, Korea, and probably California). Got home from the movie and started to port Everydevel to Apache 2.2 and MySQL 5.0. No one has bothered to do either and I am getting tired of continuing to set it up on old application stacks. The Apache part took a minor amount of reading just because of the changes in mod_perl2 from mod_perl. It was minor and mostly annoying stuff. I also setup Apache to just use pre-fork since I am personally not sure that I consider a threaded version of Apache with mod_perl to be safe. Watched Fire Fly while coding. …