We noticed some performance issues on one of our Netapp Filers and during my investigation I noticed a few things that happen under the covers and I wanted to share them with you.

Netapp SnapMirror is an asynchronous method of mirroring files from one location to another. At my company we use it to replicate property images from one datacentre to another. I've always wondered what happens to the replica as SnapMirror is a block copy though obviously the destination may be different therefore it needs to do something 'smart' to correct the differences between the two filers.

The 'smart'ness is what is causing performance problems - so lets go through them.

Here is some of the output of a 'statit' on the SnapMirror replica. The filer is idle - its not doing any traffic and its not snapmirroring.

nas02*> statit -e

Hostname: nas02 ID: xxxx Memory: 16376 MB
NetApp …

MySQL database dumps can be automated via a CRON job, without providing a password, to achieve the same follow these steps

Edit you my.cnf and make the following entries

[mysqldump]

User=root

Password=secret

Save and exit the my.cnf file

Now write the following script to the CRON job

#! /bin/sh

mysqldump –u root –all-databases –routines

>/var/lib/mysqldump/mysql.dump

Automating the database dumps without using the password will make the maintenance easier.

-Death

…

I’m hearing a lot of “NoSQL” these days. To really understand how (and) does it works, I decided to give a try on MongoDB. MongoDB (hu*mongo*us) is an open source, scalable, high-performance, schema-free, document-oriented database written in the C++ programming language. MongoDB is not a Relational Database Management System. The database manages collections of JSON […]

I often hear suggestion to use fadvise system call to avoid caching in OS cache.
We recently made patch for tar, which supposes to create archive without polluting OS cache, as like in case with backup, you do not really expect any benefits from caching.

However working on the patch, I noticed, that fadvise with FADV_DONTNEED, does not really do what I expected (I used this call as it is often suggested for this purpose). In fact it does not prevent caching, it only releases already cached data.

And if we do man fadvise, it says exactly:
FADV_DONTNEED
Do not expect access in the near future. Subsequent access of pages in this range will succeed, but will
result either in reloading of the memory contents from the underlying mapped file or zero-fill-in-demand
pages for mappings without an underlying file.

So it is …

Om, of course, got the iPad a day early. He did an unboxing post. See also: Raanan’s post on the making of the WordPress iPad app.

Weirdly, every site I visited looked great, except this one. What’s the dealio?

Here are some photos of the iPad and its excited users:

…

Answering questions asked on the site.

Andrew Stillard asks:

I have a store which will hold around 50,000 products in a products table. Each product will have 14 options, giving 700,000 options in total. These are held in an options table which is joined via the product id.

Users search for products based on the options via an Advanced Search menu.

The users need to be able to select multiple options upon which to query. I would normally use a JOIN if it was just the one option to select upon, but because its a variable number i thought it would be best to loop through the WHERE EXISTS statement.

The issue i have currently is that the query is taking a minimum of 18 seconds (And that was a query when the tables only had a fraction of the total products in).
If you can help us speed this up, or suggest an …

We are proud to announce Beta 7 (5.2.17).
As you will see, for Beta 7 we focused on UI and usability.

This Beta includes:

• Fixes for 69 bugs.
  • P1 -4, P2 – 21 , P3 – 41 and P4 – 3.
• New Windows UI components
• New “look” for windows version
• Modern style tab-controls
• New UI, toolbars and sidebar-placements
• Easier to use
• Improved look and feel
• Distinct User Snippets
• Pre-Loaded DML and DDL Snippet Libraries
• SQL Editor
  • All DDL Statements for changes are available.
  • Results menu bar
• In Output
  • added Details Pane for Action and Message

And there’s more, so check it out and see for yourself.

For more on these new features check out …

You are using MySQL Cluster and crazy enough to digest NDB API? Sick of SQL? Here's a treat: a function to make C/C++ strings ready for inserting into a VARCHAR field. The special thing about them is that the length is prefixed in the first 2 bytes.

void make_ndb_varchar(char *buffer, char *str)
{
  int len = strlen(str);
  int hlen = (len > 255) ? 2 : 1;
  buffer[0] = len & 0xff;
  if( len > 255 )
    buffer[1] = (len / 256);
  strcpy(buffer+hlen, str);
}

Yes, you can use memcpy. Whatever floats your boat.

Lets use this function for a table t1, defined as follows (note: latin1!):

CREATE TABLE t1 (
  id INT UNSIGNED NOT NULL,
  vc VARCHAR(128),
  vclong VARCHAR(1280),
  PRIMARY KEY (id)
  ) ENGINE=NDB DEFAULT CHARSET=latin1

Here is part of the code, simplified for this post:

char vc[128+1]; // Size of 'vc', +1 for length …

At the UC, I will be moderating a BoF session on the History of Database Systems on Tuesday evening at 19:00 PM. I plan and hope that this will be an open discussion, the topics I would want to discuss are:

• The history (I will prepare a few things myself in this area).
• The current state of things (SQL, the state of the "old" technologies (Network, File bases, Hierarchical) etc.
• "Failed" technologies (ORDBMS, Object databases etc).
  
• What's coming down the road (Specialized database systems, NoSQL, BI orieted systems, Column based stores with SQL or not)

I would be glad f you would attend, I'd really like this discussion and subject, and after it, I'd enjoy a beer in the bar, but that was obvious I guess.

/Karlsson

If you think I’ve been a long time in replying to your e-mails in the past month or so, there’s a good explanation for that.

For the past many years, I’ve been using an e-mail hosting provider that prides itself on offering a special class of services. Things have actually been very good, with few hiccups along the way. However, for the past month, things have seriously gone downhill with e-mails taking hours or even days to arrive.

And the problem is indeed with the provider: The Received: headers tell their own very sad story on just how long it takes for an e-mail to progress through their internal systems.

Worse, the support people are being less than helpful, refusing to recognize the problems and doing nothing to communicate what they are doing to resolve their problems.

Below is a quick analysis of just how things have been in March. The times shown are from the time an e-mail first enters the …