I’ve been seeing a few occasions where Neil J. Gunther’s Universal Scalability Law doesn’t seem to model all of the important factors in a system as it scales. Models are only models, and they’re not the whole truth, so they never match reality perfectly. But there appear to be a small number of cases where systems can actually scale a bit better than linearly over a portion of the domain, due to what I’ve been calling an “economy of scale.” I believe that the Universal Scalability Law might need a third factor (seriality, coherency, and the new factor, economy of scale). I don’t think that the results I’m seeing can be modeled adequately with only two parameters.

Here are two publicly available cases that appear to demonstrate this phenomenon: Robert Haas’s

  [Read more...]

Back in 2010 I stopped buying test servers from Dell and began building them from components using Intel i7 processors, X58-based mother boards, and modular power supplies from Ultra.  It was a good way to learn about hardware.  Besides, it was getting old to pay for Dell desktop systems with Windows, which I would then wipe off when installing Linux.  Between the educational value of understanding the systems better, selecting the exact components I wanted, and being able to fix problems quickly, it has been one of the best investments I have ever made.  And it didn't cost any more than equivalent Dell servers.

For this reason, a couple of recent articles about computer hardware caught my attention.  First, Dell is losing business

  [Read more...]

1. Apache and MySQL Logging with Syslog-ng
This article shows how to use the popular system logging tool Syslog-ng to log Apache and MySQL events. Apache does not log via syslog-ng by default so we go over two methods of easily remedying this. We also show how to use SQL queries to view syslog-ng data.

2. Using M3 to take System Monitors to the Next Level
Monitis provides built in functionality to monitor a wide variety of system statistics as well as the ability to create custom system monitors. Monitis Monitor Manager, or M3 for short, allows you

  [Read more...]

As Giuseppe Maxia recently posted we released Tungsten Replicator 2.0.4 this week.  It has a raft of bug fixes and new features of which one-line installations are the single biggest improvement.  I set up replicators dozens of times a day and having a single command for standard cluster topologies is a huge step forward.  Kudos to Jeff Mace for getting this nailed down.

So what's next?  You can get see what we are up to in general by looking at our issues list.  We cannot do everything at once, but here are the current priorities for Tungsten Replicator 2.0.5.

• Parallel replication speed

  [Read more...]

Reliable Internet access is a long-standing problem for road warriors visiting foreign countries.  Open source developers in particular have problems reconciling travel with addiction to high-bandwidth network access from laptop computers.  Wi-Fi hotspots are scarce, costly, often slow, and in some cases complicated by inconvenient local laws like Italy's Pisanu Decree.  International mobile network access plans are ridiculously expensive or like DROAM have download limits that make them useless for serious programming.

The best solution in many cases is to look for a local pre-paid mobile access plan in each country you visit.   Mobile networks are widely available and fast in developed regions, and there are cheap

  [Read more...]

I’ve been accepted to present at the brand-new and very exciting Postgres Open 2011 about system scaling, TCP traffic, and mathematical modeling. I’m really looking forward to it — it will be my first PostgreSQL conference in a couple of years! See you there.

Related posts:

Postgres folks, consider the 2011 MySQL conference

O’Reilly MySQL 2011 conference CfP is open

My sessions at the O’Reilly MySQL Conference 2011

  [Read more...]

I love short and consist install instructions. I know this is a MySQL blog but our good friend PostGreSQL has a great GIS library. This is what I learned upgrading our PostGIS system to GIS 1.5. Much thanks to Jeremy Tunnell for give this document it’s start.

Start with CentOS 5.6 x86_64 basic install.

Add the PostgreSQL Yum repository to your system.

 $ wget http://yum.pgrpms.org/reporpms/9.0/pgdg-centos-9.0-2.noarch.rpm
 $ rpm -i pgdg-centos-9.0-2.noarch.rpm

Another location for these is DAG. I have to tried these so your results may very.

You will need to exclude the packages CentOS provide by added two lines to the BASE and UPDATE sections of /etc/yum.repos.d/CentOS-Base.repo. They are:

exclude=postgresql*

  [Read more...]

What will be first? The new and updated MySQL certification or the new OSS-DB exam which is announced by LPI in Japan?

The OSS-DB is only for PostgreSQL for now, but will cover more opensource databases in the future.

There seem to be two levels:

• Silver: Management consulting engineers who can improve large-scale database


• Gold: Engineers who can design, development, implementation and operation of the database

The google translate version can be found here. I found this info on Tatsuo Ishii's blog

The Open Database Camp 2011 opens today with the Welcome Party, starting today at 7pm CEST. The party (with good Italian food and drinks) is open to all the ones who have registered in the Attendees list.
By car you have to reach Pula, take Via Nora (Nora Street), than Via Sant'Efisio (Sant'Efisio Street), until the end, directly to the party location.
Organisers will also make a bus available on Friday 6 May, leaving from Pula Hotels (Nora Club Hotel - Villa Madau - Baia Di Nora - Is Molas - Marin Hotel - Is Morus Hotel) around 18:30 and reaching

  [Read more...]

Intro

Open source Tungsten Replicator for MySQL enables multi-master, parallel and even heterogeneous replication, together with zero-down time schema upgrades, transaction filtering and more, but all these features are hardly available for another elephant in the room - i.e. PostgreSQL. At least, that was true until now.

These article series present my recent extension to Tungsten Replicator, which enables logical replication for PostgreSQL and, effectively, opens up all the advanced




  [Read more...]

Thanks to my colleague Linas Virbalas, Tungsten Replicator has just taken the next step to support full logical replication for PostgreSQL.  Linas posted an article today on his new blog describing PostgreSQL logical replication using SLONY triggers.  I saw a demo of his implementation and was really impressed.   For more information you should read the article, which provides an excellent description of how Tungsten replicates from SLONY logs.

It is pretty exciting whenever Tungsten replicates data to or from a new DBMS type, but PostgreSQL logical replication is really special.  Tungsten Replicator has been able to manage native

  [Read more...]

My Colleague Linas Virbalas has just crossed the boundary between real and virtual and has started a blog, titled Flying Clusters.
Linas is a gifted developer who is taking care of the special projects. One of such projects is replication between MySQL and PostgreSQL, which works quite well.Another project, which has just started, is about providing PostgreSQL with Advanced Logical Replication using Tungsten replicator. As you probably know, recent versions of PostgreSQL can do physical
  [Read more...]

GreenSQL invites you to participate in our May Webinars
MAY 18- Securing Databases in Minutes with GreenSQL Express
MAY 24 – Unified Database Security, the Next Generation of Database Security
Press here to sign
http://hosted.verticalresponse.com/579426/4aa0167718/316941501/bdea25b57a/

The following quotes are the first sentences in the replication chapter of two similar books. Both are admin cookbooks. One is for PostgreSQL, one for MySQL.

Replication isn't magic, though it can be pretty cool. It's even cooler when it works, and that's what this chapter is all about.
Replication requires understanding, effort, and patience. There are a significant number of points to get right. My emphasis here is on providing simple approaches to get you started, and some clear best practices on operational robustness

PostgreSQL 9 Admin Cookbook

Replication is an interesting feature of MySQL that can be used for a variety of purposes. It can help to balance server load across multiple machines, ease backups, provide a

  [Read more...]

The Open Database Camp 2011 is near. In 9 days, the welcome party starts, and then the conference itself gets going.
If you are coming earlier than Friday, May 6th, you can either use public transportation or book a private seat with a volunteer in the car pooling page. Please help the organizers: post your arrival and departure dates and times, so we may be able to help you even outside the official conference days.
About the conference itself, as everyone should know, it's a

  [Read more...]

The MySQL Conference is over. There have been many new developments, and the ones who have missed it will probably want to get a summary of the excitement, possibly from the people who have contributed to shaping the news.
The Open Database Camp will give users an opportunity to catch up. Especially to open source users in Europe.
Come and share the fun. There will be talks on MySQL, PostgreSQL, several NoSQL products, and a bunch of other cool stuff.

The Open Database Camp 2011 is barely one month away.

Sponsorship

Many thanks to all the sponsors! We very much appreciate your support.
Speaking of what, here is some important information for sponsors: The venue owners, Sardegna Ricerche, has given us the availabilkity of an ample hall for sponsors, where they can showcase their products and services.
Each sponsor will have a desk, and a double panel sized cm 195 x 75 (6.3 x 2.4 feet).

Culture, fun, and more

The Science park is something unique that geeks may want to visit. It is one of the biggest research centers in Europe, and the owners have graciously organized a guided tour before and after the conference.
Near the conference there is Nora, an





  [Read more...]

Well day two here at PgEast has drawn to a close and it was another
very informative day.

Today I concentrated on the more common tasks of a Pg DBA so I attended three
talks (four if you count mine) that where rather heavy on the technical side of being a Pg DBA

Keven Kempter drew me back again with his excellent talk on Backup and recovery methods
this time giving some very good advice on how to use and abuse of pg_Dump_all and
PG_restore. He also touched on three different recipes PITR on ProstgreSQL and gave some handy
advice on when and why to use it.

I also caught another Mongo talk this time by Steve Francia it was on the application of Mongo
in a real world web retail store. He presented a very convincing argument for the NoSQL side of things in
the retail realm namely that RDBMS works great when you have but a few similar

  [Read more...]

I guess I brought the snow with me to New York as I awoke to a nice 10cm dump. Anyway today would best be described as a day of ‘Disruptive Tech’

I first attended Kevin Kempters intro into PorstgreSQL High Availability. A very well balanced presentation that gave a very good overview of what is available out of the box for both Warm Standbys and Hot Standbys how they can be very easily implemented. He also gave a quick overview of other tools that can be used including Slony for detailed fail-overs and PgPool for load balancing and relication. Not very disruptive but it does show that Pg is on par with most of the heavy hitters such as MySQL and Oracle.

The keynote this year was by Ed Boyajian the CEO or EnerpriseDB and he gave an big picture of the DB in terms of market which is a whopping 26$ Billion a year in the US alone of which the the two five players have 90% of the

  [Read more...]

“I believe that one should benchmark before making any technology decisions.” An interview with Pieter van Zyl creator of the OO7J benchmark. In August last year, I published an interesting resource in ODBMS.ORG, the dissertation of Pieter van Zyl, from the University of Pretoria:“Performance investigation into selected object persistence stores”. The dissertation presented the OO7J [...]

Hi Everyone,

I would like to personally invite you to a GreenSQL Express Webinar,
I’ll be demonstrating GreenSQL Express, the free and simple way to keep your information private and safe.

On Wednesday, March 16th (just 2 weeks from now),
It’s called “How to Protect Sensitive Information in Minutes: Setting up GreenSQL Express with Basic Security Rules”

If you’re serious about protecting your data, you need to hear and see how it’s done. I’ll talk about:

1. Why you need a Database firewall / security solution
2. Where and How to install GreenSQL Express in your infrastructure
3. How to use GreenSQL Express to protect you database
4. How to create the security polices you need in minutes
5. How to protect your database from SQL injection attacks
6. How to

  [Read more...]

From the Security threat report 2011 by Sophos, Page 46:

“Cybercrime is encroaching more and more into the business space. Industrial espionage, spearphishing of important employees to breach network boundaries and mass theft of customer information are more diffcult to detect and have very serious consequences. At the same time, network boundaries are becoming ever more indistinct and porous as new technologies enable greater access from remote workers and mobile devices. In addition, legal requirements place greater emphasis on traceability and compliance with predefned standards of data hygiene.

Increasing amounts of sensitive data is stored, accessed and manipulated in databases connected to company websites as businesses increasingly interact with their customers

  [Read more...]

PostgreSQL 9.0 High Performance

PostgreSQL 9.0 High Performance. By Gregory Smith, Packt 2010. About 420 pages. (Here’s a link to the publisher’s page for this book.)

I enjoyed this book a lot and recommend it to everyone who uses PostgreSQL or MySQL. MySQL users should benefit from understanding PostgreSQL. Beyond that, I learned a lot from this book that I can apply directly to MySQL. In particular, the book begins with a few chapters on hardware performance, benchmarking, and configuration. This material is

  [Read more...]

Giuseppe has a great post about the Evolution of MySQL metadata, and I thought I’d have a look at what we have in Drizzle. It’s pretty easy to work out how many tables are in each schema, we just query the standard INFORMATION_SCHEMA.TABLES view:

drizzle> select table_schema,count(table_name)
    ->  from information_schema.tables
    -> group by table_schema;
+--------------------+-------------------+
| table_schema       | count(table_name) |
+--------------------+-------------------+
| DATA_DICTIONARY    |                53 |
| INFORMATION_SCHEMA |                20 |
+--------------------+-------------------+
2 rows in set (0 sec)

In Drizzle it’s important to note that

  [Read more...]

I’ve just committed some changes to diskstats, an I/O analysis tool in Aspersa that’s actually been in the Subversion repository for a long time, but in a barely usable fashion and with no documentation. Now it’s usable and documented.

It is basically a reimplementation of iostat in awk. Why on earth would I reinvent that wheel? Because I spend a lot of time gathering and analyzing raw data from /proc/diskstats, which is vital to really understanding what the storage subsystem is doing. The iostat tool hides important details. Seeing that detail has immediately solved many a disk performance problem and proven SAN vendors wrong, for instance. (I used to do this the old-fashioned

  [Read more...]

If you execute the following, what does your RDBMS do?

CREATE TABLE t1 (a int);
START TRANSACTION;
INSERT INTO t1 (a) VALUES (1);
START TRANSACTION;
INSERT INTO t1 (a) VALUES (2);
ROLLBACK;
SELECT * FROM t1;

The answer may surprise you.

A couple of extremely informative recent blog posts have gone to either Planet MySQL or Planet PostgreSQL, but not both, and I think everyone on both aggregators who cares about database internals should be interested in them. Here they are:

• Robert Haas: MySQL vs. PostgreSQL, Part 2: VACUUM vs. Purge
• Ewen Fortune: How InnoDB handles REDO logging

Happy cross-pollination.

Related posts:

Awesome Postgres/MySQL cross-pollination

Why is MySQL more popular

  [Read more...]