The latest from the MySQL community

In the press in the last two days has been the reported outage of Amazon Web Services Elastic Compute Cloud (EC2) in just one North Virginia data center. This has affected many large website includes FourSquare, Hootsuite, Reddit and Quora. A detailed list can be found at ec2disabled.com.

For these popular websites was this avoidable? Absolutely.

Basic scalability principles if deployed in these systems architecture would have averted the significant downtime regardless of your development stack. While I work primarily in MySQL these principles are not new, nor are they complicated, however they are fundamental concepts in scalability that apply to any technology including the …

VMware launches Cloud Foundry. Red Hat heads for Ceylon. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca, and daily at Paper.li/caostheory
“Tracking the open source news wires, so you don’t have to.”

# VMware launched Cloud Foundry Platform-as-a-service and open source project.

# Red Hat’s Gavin King revealed details of the company’s Ceylon project.

# Red Hat submitted a number of specification requests for Java EE 7.

# Terracotta accused Red Hat of “trying to pull a fast …

I chatted today about VMware's Cloud Foundry with Roger Bodamer, the EVP of products and technology at 10Gen. 10Gen's MongoDB is one of three back-ends (along with MySQL and Redis) supported from the start by Cloud Foundry.

If I understand Cloud Foundry and VMware's declared "Open PaaS" strategy, it should fill a gap in services. Suppose you are a developer who wants to loosen the bonds between your programs and the hardware they run on, for the sake of flexibility, fast ramp-up, or cost savings. Your choices are:

An IaaS (Infrastructure as a Service) product, which hands you an emulation of bare metal where you run an appliance (which you may need to build up yourself) combining an operating system, application, and related services such as DNS, firewall, and a …

Well day two here at PgEast has drawn to a close and it was another
very informative day.

Today I concentrated on the more common tasks of a Pg DBA so I attended three
talks (four if you count mine) that where rather heavy on the technical side of being a Pg DBA

Keven Kempter drew me back again with his excellent talk on Backup and recovery methods
this time giving some very good advice on how to use and abuse of pg_Dump_all and
PG_restore. He also touched on three different recipes PITR on ProstgreSQL and gave some handy
advice on when and why to use it.

I also caught another Mongo talk this time by Steve Francia it was on the application of Mongo
in a real world web retail store. He presented a very convincing argument for the NoSQL side of things in
the retail realm namely that RDBMS works great when you have but a few similar products
such as books, CDs and …

I guess I brought the snow with me to New York as I awoke to a nice 10cm dump. Anyway today would best be described as a day of ‘Disruptive Tech’

I first attended Kevin Kempters intro into PorstgreSQL High Availability. A very well balanced presentation that gave a very good overview of what is available out of the box for both Warm Standbys and Hot Standbys how they can be very easily implemented. He also gave a quick overview of other tools that can be used including Slony for detailed fail-overs and PgPool for load balancing and relication. Not very disruptive but it does show that Pg is on par with most of the heavy hitters such as MySQL and Oracle.

The keynote this year was by Ed Boyajian the CEO or EnerpriseDB and he gave an big picture of the DB in terms of market which is a whopping 26$ Billion a year in the US alone of which the the two five players have 90% of the market one having more than half.

He made …

Hi Folks,

Now that we’re blogging again I thought I might as well continue to do so.

Today we’re reading data from MongoDB with Pentaho Data Integration.  We haven’t had a lot of requests for MongoDB support so there is no step to read from it yet.  However, it is surprisingly simple to do with the “User Defined Java Class” step.

For the following sample to work you need to be on a recent 4.2.0-M1 build.  Get it from here.

Then download mongo-2.4.jar and put it in the libext/ folder of your PDI/Kettle distribution.

Then you can read from a collection with the following “User Defined Java Class” code:

import java.math.*;
import java.util.*;
import …

Given the popular response from my first lesson in improving scalability where I detailed simple ways to eliminate unnecessary SQL, let me share another common bottleneck with MySQL scalability that can be instantly overcome.

Analyzing the writes that occur on a system can expose obvious potential bottlenecks. The MySQL Binary Log is a wealth of information that can be mined. Simple DML Counts per table can be achieved by a single line command.

Let’s look at the following example output of a production system:

mysqlbinlog /path/to/mysql-bin.000999 |  \
   grep -i -e "^update" -e "^insert" -e "^delete" -e "^replace" -e "^alter"  | \
   cut -c1-100 | tr '[A-Z]' '[a-z]' |  \
   sed -e "s/\t/ /g;s/\`//g;s/(.*$//;s/ set …

I’ll be presenting several sessions at the O’Reilly MySQL Conference & Expo 2011, which is April 11-14 in Santa Clara, California. I recommend this conference to anyone interested in open-source databases including MySQL, PostgreSQL, CouchDB, MongoDB, and others. There is very good coverage of a diverse list of open-source databases.

My sessions are as follows:

• Building on Strengths, Learning from Differences: a keynote address on the open-source database ecosystem, how we got here, and what we can do to make the most of the future.
• Forecasting MySQL Performance and Scalability: mathematical models for forecasting performance and …

I just received a confirmation that my presentation proposal for the MySQL user conference 2011 was accepted! The title for my proposal is MQL-to-SQL: a JSON-based Query Language for RDBMS Access from AJAX Applications, and it covers pretty much everything implied by the title.

As always, the Hyatt Regency Hotel in Santa Clara, California serves as the venue. The conference will be held from April 11-14. Except for the venue and period, I think this year's conference will bear few similarities to previous editions. Let me try and explain.

This year's theme is "MySQL, the ecosystem and …

You may have seen my posting regarding "eventual consistency" some months ago, and you may have come to the conclusion that I was insisting that a SQL based RDBMS is the way to go for just about anything. Tell you what, that is not so. And nether am I against using. say, MongoDB, where it is appropriate.

The whole deal with Eventual consistency is something that I am still opposed to, I want to know if my data is consistent. And I am not not sure that you cannot have a fully consistent, distributed system either. But I guess that debate goes on. And I still want my base data to be consistent. Like in RDBMS-SQL-Foreign-keys-all-over-the-place-and-not-a-bl**dy-bit-lost-in-the-MyISAM-swamp consistent. That is what I want the base data to look like. And if there are compromises …