I am happy to announce the release of mylvmbackup version 0.8. mylvmbackup is a tool for quickly creating backups of a MySQL server's data files. To perform a backup, mylvmbackup obtains a read lock on all tables and flushes all server caches to disk, makes an LVM snapshot of the volume containing the MySQL data directory, and unlocks the tables again. The snapshot process takes only a small amount of time. When it is done, the server can continue normal operations, while the actual file backup proceeds.

Below is the list of changes since version 0.6. You may wonder what happened to version 0.7 - it had a rather short life cycle as I was informed about a bug that I fixed quickly before I made a wider release announcement of 0.7.

• Fixed a bug in the InnoDB recovery function: the second mysqld process …

Earlier this week I attended the Linux Foundation Collaboration Summit, serendipitously held here Austin.  It was a fantastic opportunity to meet a lot of the people in the community whom I hadn't met before as well as to catch up with old friends.

 
Maddog (R) helping Executive Director Jim "Led" Zemlin to Flourish 

Great Speakers
(and I'm not talking Bose, which really aren't great speakers anyway)

There was an impressive line up of panels and speakers the first day of the summit …

“The question is not why use open source, but how to best use open source,” wrote Matt Asay earlier this week. It was a throwaway point but one that I think deserves more attention.

It occurred to me that “how” rather than “why” is the most significant question that open source vendors and projects should be answering right now as they try to encourage greater adoption of open source software.

There can’t be a CIO or IT director left on the planet that hasn’t either asked or been told why they should deploy open source software. They are either inclined towards believing the claims of theoretical benefits or they’re not. How many have asked or been told how they can take advantage of open source software?

Certainly those that are convinced or intrigued by the potential benefits will have gone on to explore …

All production servers are normally installed using kickstart files. Unfortunately, the ks file had a bug that it didnt add a swap partition if there was only one logical or physical disk.

Once the database was setup without swap, the configuration was sized such that all processes fit into memory. But we started seeing OOM killing processes due to lack of memory. Though under normal conditions we had 1-2 GB of free memory.

Unfortunately, OOM at times picked to kill mysql. We were running 500GB+ database having myisam tables. We ended up having to repair the tables every time OOM felt like killing mysqld . Adding a swap device might help us fix the issue, but for that we might have to resize the logical/physical partitions, which wasn’t cool. Then this following idea popped up, create a file, format it as swap and add it as swap device dynamically and also add it /etc/fstab to survive reboot.

dd …

I recently was interviewed by Packt Publishing for their Impackt '08 web pages:

Ever since the formal adoption of the term in 1998, Open Source has experienced growth and adoption rates that defy pressures and suggestions that it’s a viable option for enthusiasts and geeks only. Governments, corporations as well as small businesses have begun to choose Open Source over proprietary software. However, with the global economy facing an uncertain future, how will open source be impacted? Can it continue to grow despite this?

With these questions in mind and more, Packt approached some people at the heart of this movement to understand their take on the future of open source.

The …

It is interesting to read RedmondDeveloper News’s take on Oracle’s attitude to open source this morning, especially this paragraph quoting Monica Kumar, Oracle’s senior director for Linux and open source product marketing:

“”We haven’t seen our customers asking for open source databases,” she told me. “Not many customers are interested in looking into the code and mucking around with it, and making changes to it. All they care about is ‘give me the best support, give me the lowest price of entry’.” For that Kumar pointed to Oracle Express.”

It is difficult to disagree with the second part of Monica’s statement. Cost savings are routinely cited as the biggest driver for open source database adoption, while the lack of robust support is the biggest barrier to open source adoption.

Certainly these were the findings of our …

Today I was asked a question about defining custom extensions for vim syntax highlighting such that, for example, vim would know that example.lmx is actually of type xml and apply xml syntax highlighting to it. I know vim already automatically does it not just based on extension but by looking for certain strings inside the text, like <?xml but what if my file doesn't have such strings?

After digging around I found the solution. Add the following to ~/.vimrc (the vim configuration file):

1
2
3

syntax on
filetype on
au BufNewFile,BufRead *.lmx set filetype=xml

After applying it, my .lmx file is highlighted:

…

There's a relatively little known feature about Linux IO scheduling that has a pretty significant effect in large scale database deployments at least with MySQL that a recent article on MySQL Performance Blog prompted me to write about. This may have an effect on other databases and random I/O systems as well, but we've definitely seen it with MySQL 5.0 on RHEL 4 platform. I have not studied this on RHEL 5, and since the IO subsystem and the Completely Fair Queue scheduler that is default on RHEL kernels has received further tuning since, I can not say if it still exists.

Though I've heard YouTube discovered these same things, …

This article contains my notes and detailed instructions on setting up a MySQL cluster. After reading it, you should have a good understanding of what a MySQL cluster is capable of, how and why it works, and how to set one of these bad boys up. Note that I'm primarily a developer, with an interest in systems administration but I think that every developer should be able to understand and set up a MySQL cluster, at least to make the dev environment more robust.

Notes

• High availability. If one or some of the machines go down, the cluster will stay up, as long as there is at least one copy of all data still present. The more redundant copies of data there are, the more machines you can afford to lose.
• Scalability. Distributed architecture allows for load balancing. If your MySQL …

We made a very significant announcement last week, of a collaboration with one of the most (if not the most) security sensitive institutions on earth, the United States government's National Security Agency. They've joined the burgeoning OpenSolaris community, to collaborate with Sun and other community members on the future of ultra-secure operating systems.

To put this in context, community engagement has always been one of the most important ways Sun innovates in the marketplace - we partner with those that have extreme demands (whether it's the world's largest supercomputing facility, or the world's most paranoid security professionals (no offense intended), or the world's largest archival storage facilities), and then we leverage that expertise to create products for the mass market. We let extreme customers teach …