Back to year 2006 when I was working for my first sphinxsearch project I was playing with stopwords files. Stopwords is basically a small set of highly frequent words you often don’t want to search for (like “I”, “Am”, “The”, etc). For most sphinx instances they only wasting index space and slower your search queries by finding all occurrences of these non-important words.

Say if you are searching for “when is jane’s birthday” you are actually looking to find documents with “jane’s birthday”, and you don’t really care about lot’s of documents (blog posts, news articles, etc) with only “when” and “is” inside.

Remove those high frequency words from search index is usually smart move and ages ago I’ve created two stopword file samples which I’m using by now.

…

This week Sheeri visits the Schooner offices and inteviews Dr John Busch, the CTO, Founder, and Chairman of the Board for Schooner. Schooner is able to fully use multi-core processors and do fully synchronous replication without losing performance. The new 5.1 release offers parallel synchronous replication with automated failover between remote data centers.

News
Both Sheeri and Gerry have new jobs - Sheeri wrote about her change at http://palominodb.com/blog/2011/11/23/last-day-palominodb

read more

News/Feedback
Last week we talked about how we learn and keep up to date with changes to MySQL. Mark Grennan of mysqlfanboy.com posted his own list of MySQL resources.

Call for papers for Percona Live: MySQL Conference & Expo 2012 is open until Monday, December 5th. To submit a paper, first register as a speaker at http://www.percona.com/live/mysql-conference-2012/user/register and then go to My Account -> Submit Proposal.

read more

The ScaleBase Analysis tool gives a schema a grade between 1 and 100 for being “sharding compatible”. It’s a neat feature, but many ask me how the grade is calculated. Well – here goes.

First of all, a good schema is one that is easy to shard. Database Normalization is usually a good thing when sharding. It means that finding the sharding key is easy, relationships between tables are clear, and the queries themselves are usually much simpler. So we try to give a grade on how well the schema is normalized.

After the sharding configuration is determined (see here on how this should be done), we review your MySQL General Log, to understand the value you can expect from the sharding configuration:

• Statements that run on sharded tables, and hit a …

One of the great things about the HTTP protocol, besides status code 418, is that it's stateless. A web server therefore is not required to store any information on the user or allocate resources for a user after the individual request is done. By that a single web server can handle many many many different users easily, and well if it can't anymore one can add a new server, put a simple load balancer in front and scale out. Each of those web servers then handles its requests without the need for communication which leads to linear scaling (assuming network provides enough bandwidth etc.).

Now the Web isn't used for serving static documents only anymore but we have all these fancy web apps. And those applications often have the need for a state. The most trivial information they need is the current user. HTTP is a great protocol and provides a way to do authentication which …

MySQL 5.6 proposes improved partition maintenance through "partition exchange". Even if not yet available, dbi services performed some tests on the MySQL Community Edition5.6.2 m5 release (still under Development).

A quick note, I am speaking at Percona live in London next week… its should be a rip roaring time. I have two topics I am speaking on.

The first is on building a MySQL Data Access Layer with Ruby and Sinatra. While this may seem a bit odd, its actually very cool and useful. With

As a followup to my previous post on installing HandlerSocket on CentOS 5 + Percona-Server 5.1, I’ve been asked to provide an updated HOWTO for RHEL 6 + Percona-Server 5.5. Although very similar to the original steps, there are some new traps (RHEL 6 comes with SELinux enabled) and there are some new ec2 gotchas! [...]

In this tutorial I will describe, how to use nmon (Nigel’s performance Monitor) to monitor performance data in the interactive mode or in the capture mode.

nmon can display / capture the following performance data

• CPU utilization
• Memory use
• Kernel statistics and run queue
• Disks I/O rates, transfers, and read/write ratios
• File systems size and free space
• Disk adapters
• Network I/O rates, transfers, and read/write ratios
• Paging space and paging rates
• Machine details, CPU and OS specification
• Top processors
• User defined disk groups
• Asynchronous I/O – AIX only
• Workload Manager – AIX only
• ESS and other disk subsystem – AIX only
• Dynamic LPAR changes …

A special extended edition of Tech Messages for 2011-08-31 through 2011-09-21:

• Lab Overview
  This guy is serious about his lab!
• Throughput
  All sorts of useful information here. My favorite is the first table which includes bandwidth and packet throughput for different kids of circuits.
• Must see: 5 Steps to Scaling MongoDB (Or Any DB) in 8 Minutes
  These tips are relevant no matter what database system you are using.
• Understanding and …