Yesterday, I covered how you can do an initial “replication” of data from MySQL to DynamoDB and how this can improve performance, and save storage space. The follow on question becomes:

That’s Great Nick. But how do I do keep my data up to date?

We’ve got data in our Airline Performance dataset through 31-DEC-2007. I loaded 1 year, all of 2007, for the previous example. What happens when the FAA publishes their 2008 January results, and we’ve loaded the new months worth of data into MySQL?

MySQL:

select count(*) from otp.ontime; 8061223
select count(*) from ontime where FlightDate > ’2007-12-31′; 605765
select count(*) from ontime where FlightDate <= ’2007-12-31′; …

I'm pleased to announce the release of Spider storage engine version 2.8(beta).
Spider is a Storage Engine for database sharding.
http://spiderformysql.com/

The main changes in this version are following.
- Add table parameter "link_status".
  You can change link_status using "alter table" statement.
  Spider's link fault management is table level.

Please see "99_change_logs.txt" in the download documents for checking other changes.

Enjoy!

I am Domas Mituzas, and I have just joined Facebook MySQL team. I've been working at MySQL Support before, as well as did Wikipedia data and performance engineering on my free time - and I've blogged about that a bit.

Here at Facebook I've started working on large scale database deployment introspection - we want to know that everything churns happily, and knowing what is wrong and why it is wrong is the first step to improving the mental and emotional state of our servers.

Now, fetch me few thousand scalpels, before I grab my axe... ;-)

We are going to release openSUSE 11.2 soon, so I want to let you know about one problem you may expect. One of the openSUSE 11.2 features is support for ext4. Well, if you will just click on next all the time during the installation, default filesystem for your root partition will be ext4. What is the problem with that? Well, ext4 in current kernel has barriers enabled by default. It's not a problem, it's a feature, but it has some unpleasant side effects which you may encounter. Let's take a closer look.

Barriers

Barriers are used to make sure that journal is really written to your filesystem and that nothing is missing. It has one big advantage - your data are safe. On the other hand it has disadvantage as well. You have to make sure that everything is written so you can't really take full advantage of all caches and such. There is a nice article on LWN.net about …

We are going to release openSUSE 11.2 soon, so I want to let you know about one problem you may expect. One of the openSUSE 11.2 features is support for ext4. Well, if you will just click on next all the time during the installation, default filesystem for your root partition will be ext4. What is the problem with that? Well, ext4 in current kernel has barriers enabled by default. It’s not a problem, it’s a feature, but it has some unpleasant side effects which you may encounter. Let’s take a closer look.

Barriers

Barriers are used to make sure that journal is really written to your filesystem and that nothing is missing. It has one big advantage – your data are safe. On the other hand it has disadvantage as well. You have to make sure that everything is written so you can’t really take full advantage of all caches and such. There is a nice article on LWN.net …

Someone recently posted this to an email list as a sample of an interesting SHOW INNODB STATUS output:

mysql> SHOW ENGINE INNODB STATUS\G
          _______  _______
|\     /|(  ____ \(  ____ \
| )   ( || (    \/| (    \/
| |   | || (_____ | (__
| |   | |(_____  )|  __)
| |   | |      ) || (
| (___) |/\____) || (____/\
(_______)\_______)(_______/

 _______  _______  _______ _________ _______  _______  _______  _______
(  ____ )(  ___  )(  ____ \\__   __/(  ____ \(  ____ )(  ____ \(  ____ \
| (    )|| (   ) || (    \/   ) (   | (    \/| (    )|| (    \/| (    \/
| (____)|| |   | || (_____    | |   | |      | (____)|| (__    | (_____
|  _____)| |   | |(_____  )   | |   | | ____ |     __)|  __)   (_____  )
| (      | |   | |      ) |   | |   | | \_  )| (\ (   | (            ) |
| )      | (___) |/\____) |   | |   | (___) || ) \ \__| (____/\/\____) |
|/       (_______)\_______)   )_(   (_______)|/   \__/(_______/\_______) 

I thought it was worth …

Yahoo! Open! Sources! Traffic! Server! Funding for 10gen. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

For the latest on Oracle’s acquisition of MySQL via Sun, see Everything you always wanted to know about MySQL but were afraid to ask

# Yahoo! Open! Sourced! Traffic! Server!

# Red Hat launched Enterprise Virtualization for Servers for managing Linux and Microsoft Windows servers.

# 10gen, the company behind MongoDB, has …

I have been asked a number of times "do you think there is a need for a column oriented database in the open source world?"

The answer has been yes!

Users and vendors have asked me this question a number of times. The problem has been most of the vendors were interested in creating closed source solutions around either Drizzle/MySQL, or, did their efforts in a way that made serious modifications to the backend (aka... made poor use of the storage engine interface).

For these reasons I have not really found myself all that thrilled to work with what has been out there. Also, I would often find that the commitment to open source was either luke warm or "we will do it, once we have some traction...".

My response to that? "Tell me more when you open source it. I'll see if it will work."

For this reason I was very happy to see Calpont do their release of …

InnoDB uses a 16kb page by default. I want to know whether performance improves with an 8kb database page for my workload. Two servers were setup to run a mirror of the production database workload. One used 8kb InnoDB pages and the other used 16kb.

It isn't clear that my performance will improve with 8kb pages. But my results are a function of my workload and data. I expect that 8kb pages will be much better for others. For example, when your data access pattern is uniform and most data is fetched by primary key, then smaller pages can be better as less space is wasted in the buffer pool per active row. However, a smaller page size will also waste more space from fragmentation for LOB columns. Peak IOPs on spinning disk are similar for 8kb pages and 16kb pages with random IO bound workloads. But when using flash, peak IOPs for 8kb pages are much higher than 16kb pages. So you, or your consultant, have interesting work to do if you want to …

Today I’m proud to announce the first public release of our ActiveRecord database connection magic plugin: DbCharmer.

DB Charmer – ActiveRecord Connection Magic Plugin

DbCharmer is a simple yet powerful plugin for ActiveRecord that does a few things:

1. Allows you to easily manage AR models’ connections (switch_connection_to method)
2. Allows you to switch AR models’ default connections to a separate servers/databases
3. Allows you to easily choose where your query should go (on_* methods family)
4. Allows you to automatically send read queries to your slaves while masters would handle all the updates.
5. Adds multiple databases migrations to ActiveRecord

Installation

There are two options when …