At the most recent MySQL conference many people asked us how we are and planning to scale MySQL. One of the ways we're planning on providing better quality of service to our users is the deployment of our caching module for Linux, Flashcache. Flashcache is a simple write back persistent block cache designed to accelerate reads and writes from slower rotational media by caching data in SSD's.

We built Flashcache to help us scale InnoDB/MySQL, but it was designed as a generic caching module that can be used with any application built on top of any block device. For InnoDB, when the working set does not fit in the InnoDB buffer pool, read latency is significantly improved due to caching more of the working set in faster media, such as SSD's. We also improve write performance by first caching writes in SSD's and lazily flushing the data back to disk.

We hope to share more information about our Flashcache deployment in the future, but …

Recently when talking to someone about PBMS it occurred to me that I had been thinking about BLOBs in the traditional database sense in that they were atomic blocks of data the content of which the server knew nothing about. But with PBMS that need not be the case.

The simplest enhancement would be to allow the client to send a BLOB request to the PBMS daemon with an offset and size to just return a chunk of the BLOB. Depending on the application and the BLOB contents this may make perfectly good sense, why force the client to retrieve the entire BLOB if it only want part of it.

A much more interesting idea would be to enable the user to provide custom server side functions that they could run against the BLOB.

So how would his work?

The PBMS daemon would provide its own "BLOB functions" plugin API. The API would be quite simple where the plugin would register the function names it …

VMware and Salesforce.com launch VMforce. Red Hat provides Cloud Access. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# VMware and Salesforce.com launched VMforce, a platform for developing and deploying Java cloud applications.

# Red Hat Cloud Access enables enterprises to use their Red Hat Enterprise Linux subscription on Amazon Web Services.

# Canonical announced Ubuntu 10.04 LTS Server Edition, Desktop Edition and ISV support.

While there is a dizzying array of technologies that have the “NoSQL” label applied to them, I’m looking for one to replace a MySQL cluster. This particular cluster has roughly a billion records in it, uses a few TB of disk space, and is growing all the time. It is currently a MySQL master with several slaves and handles a fairly light query volume. The reasons we’d like to change it are:

• ALTER TABLE takes an unreasonably long time, so we can’t add or removed indexes or columns. Changes take over a month.
• The data footprint is large enough that it requires RAID and we seem to be really good at breaking RAID controllers.
• Queries are not very efficient, due partly to the underlying data organization, and due partly to the sheer amount of it compared to the available RAM.
• The data really isn’t relational anymore, so a document store is more appropriate.  It’s just that when it was set …

SPJ (preview, not production ready) is a new feature allowing some types of JOINs to be pushed down and executed inside the data nodes! This allows for, in many cases, much faster JOIN execution.

Now we would love to get your feedback on this new feature:

• Does what we have right now improve performance for you?
• Are there other types of JOINs we should support in order to improve performance in you application (currently only eq_ref is supported)?
• What application are you using?

There are some limitations currently:

• node failure handling of SPJ is not complete, so if a data node crash, there are side-effects.
• only eq_ref is supported - other JOINs are executed as normal.
• Don't put this in production.
  

Obtaining the MySQL Cluster SPJ preview version:

• You must know how to build MySQL Cluster …

As requested by a few fans out there, here are the slides of my presentation:

Pentaho Data Integration 4.0 and MySQL.pdf

I had a great time at the conference, met a lot of nice folks, friends, customers, partners and colleagues. After the conference I was unable to get back home like so many of you because of the Paul Simon singing Eyjafjallajökul volcano in Iceland.

So I ended up flying over to Orlando for a week of brutal PDI 4.0 RC1 hacking with the rest of the l33t super Pentaho development team.  However, after 2+ weeks from home, even a severe storm …

Coming from a great MariaDB contributor, Mark, is:

1. MariaDB 5.1.44 / 5.2.0 Beta Binaries for Solaris 10 SPARC, and Debian GNU/Linux SPARC. Mark does a fabulous job of building these binaries, and he does them really quickly. If you’re on the SPARC platform, give it a go. Send some feedback, also.
2. Mark has also spent some time developing virtual machines. All you need to get started is download VirtualBox. Mark provides an OpenSolaris 0906 + MariaDB 5.1.44 VM as well as an …

When I celebrated my birthday, I got the idea that there must theoretically be a date on which two peoples ages are in a certain ratio. The basic idea were born when I thought that there must be a date on which my mother is exactly twice as old as me. Actually, this is the point of time when I'm as old as my mother was, when I were born. That means

Read the rest »

For all of those linux users out there that have moved over to, or tried out, Solaris10 or OpenSolaris because they heard the tales of how MySQL is faster on Solaris… or perhaps you wanted to learn how to use Sol10 for the great features of Zones or the ZFS filesystem? Regardless of why you’re on it you are probably wondering why Linux has colored output of filenames and directories but Solaris does not. The question of ‘why?’ isn’t important, but how to enable colors is. It’s very simple, and here’s how I fixed it. This is a result of digging through multiple semi-related links on Google.

1. Download all packages from SunFreeware.com
   • dependency: libintl-3.4.0-sol10-x86-local
   • dependency: libiconv-1.13.1-sol10-x86-local
   • dependency: gmp-4.2.1-sol10-x86-local
   • dependency: gcc-3.4.6-sol10-x86-local or libgcc-3.4.6-sol10-x86-local depending on your …

MySQL 5.1 still suffers to some degree from reduced performance under high concurrency. Until MySQL 5.5 is ready for production, we need some way constrain the amount of work we try to do in parallel. innodb_thread_concurrency is one control we have available, but it isn't perfect. When threads are blocked on IO, for example, no other thread can reuse its concurrency slot and try to accomplish work in the meantime. So most of us probably set innodb_thread_concurrency higher than the number of CPUs our host has, and possibly set it to 0 (unlimited).

Another way to control concurrency is limiting the number of client connections within MySQL by setting the max_connections variable. Usually we want to allow clients to stay connected longer than the duration of a single query to avoid having to reconnect for the next one so we set a connection limit of few thousand. We expect our clients to disconnect in a timely fashion …