There are two search engines that have promise that made themselves public in 2010: DuckDuckGo and Blekko. DuckDuckGo has active search spam removal, you can access it via secure HTTP (HTTPS), and is a search engine that also relies on crowd sourced data.

Web of Trust has reputation ratings of over millions of websites, and has an active community of about 15 million users now. Best of all, there’s no bots doing these ratings, but community members (trust metrics are crowd sourced).

DuckDuckGo and Web of Trust have a partnership now, so you can simply change the settings to display WoT ratings instead of the favicons when …

MySQL Full Text Search Limitations

Suppose you have a MyISAM table containing a column with a full text index. This table starts to grow to a significant size (millions of rows) and gets updated fairly frequently. Chances are that you’ll start to see some bottlenecks when accessing this table, since without row level locking, the reading and writing operations will be blocking each other.

A solution that many people would suggest right away is to use the master for writes and a slave for reads, but this only masks the problem, and it won’t take long before enough read traffic on the slave starts causing slave lags.

Why Sphinx?

The main difference between the Sphinx search engine and other alternatives is its close integration with MySQL. For example, it can be used as a storage engine.  In this way, Sphinx’s impact on existing application code …

Searching for MySQL with WolframAlpha, the latest entry in the search engines arena, I had a surprise:

Interpreting "mysql" as "mycil"
Input interpretation:
 chlorphenesin

A further search for chlorphenesin, explains that it is a drug pertaining to the "central muscle relaxants" category.

I recently got this email from a reader on my site and since I haven't posted for a while, I thought it might be a good discussion:

Jay,

I've been reading your site and had a question that you might have good insight for.  

I'm working on a high-traffic website that includes forums.  All the code is custom PHP, including the forums, simply because the PHP BB software out there doesn't scale as large and as well as I wanted.  I've implemented heavy memcached usage, distributed databases, etc. to handle any level of growth we may hit.

read more

I recently got this email from a reader on my site and since I haven't posted for a while, I thought it might be a good discussion:

Jay,

I've been reading your site and had a question that you might have good insight for.  

I'm working on a high-traffic website that includes forums.  All the code is custom PHP, including the forums, simply because the PHP BB software out there doesn't scale as large and as well as I wanted.  I've implemented heavy memcached usage, distributed databases, etc. to handle any level of growth we may hit.

read more

I’ve been coding a couple of scripts that run on 5 minute intervals to grab RSS/Atom feed data from http://mysql-dba.com and import that into a MySQL database. It idea is to create a search function for the site that will look at all past data from the aggregated feeds. Since there are multiple pollers running at different intervals I decided to use Innodb for the read/write nature of the poller/processing scripts.

This is very simple so far - and as such I felt it should be documented from the start unlike many of my other projects. Here’s the feed table that is storing the information from the RSS feeds.


mysql> show create table feed_items\G
*************************** 1. row ***************************
Table: feed_items
Create Table: CREATE TABLE `feed_items` (
`id` bigint(20) NOT NULL auto_increment,
`rss_site_id` int(11) …