MySQL Full Text Search Limitations

Suppose you have a MyISAM table containing a column with a full text index. This table starts to grow to a significant size (millions of rows) and gets updated fairly frequently. Chances are that you’ll start to see some bottlenecks when accessing this table, since without row level locking, the reading and writing operations will be blocking each other.

A solution that many people would suggest right away is to use the master for writes and a slave for reads, but this only masks the problem, and it won’t take long before enough read traffic on the slave starts causing slave lags.

Why Sphinx?

The main difference between the Sphinx search engine and other alternatives is its close integration with MySQL. For example, it can be used as a storage engine.  In this way, Sphinx’s impact on existing application code …

I just filed a very annoying bug when trying to compile with plugin engines using the 5.1.xx source tarball.

Description

I am trying to test SphinxSE as a plugin instead of getting it statically linked and came across an annoying bug. When using the configure --with-plugins option only once, the engine is statically linked. When using it twice, the first engine is created as a plugin, and the 2nd one is linked statically. Here are a couple of examples:./configure –prefix=/usr/local/mysql-5.1.33 –with-plugins=innobase –with-plugins=sphinx

plugin_innobase_shared_target='ha_innodb.la'   <-- plugin
plugin_innobase_static_target=''
plugin_sphinx_shared_target=''
plugin_sphinx_static_target='libsphinx.a'       <-- static
./configure --prefix=/usr/local/mysql-5.1.33 --with-plugins=sphinx …

As you may know mysql fulltext search is not highly scalable.  One of the options to get around this scalability limitation, which I prefer, is to use Sphinx.  You can use Sphinx with out having to alter your mysql installation.  But, if you would like to use from within mysql and not have to worry about how to pass data between Sphinx and MySQL, you can enable sphinxse (sphinx storage engine).  It is not included with mysql by default so you will have to compile it yourself.

Here are the instructions on how to get sphinxse compiled with your mysql installation on CentOS x64.  I am sure same instructions will work for other flavors but I have not tested it.  I will be compiling the most current version of sphinx (0.9.8) with most current stable version of mysql (5.0.51b) at the time of the writing.  Let’s get the appropriate …