What a great day! I see that the drizzle team has merged in my (and Padraig O'Sullivan's) memcached Functions for Drizzle (UDFs). I'm really glad to have this in drizzle now as it adds a means of interacting with memcached from within Drizzle. I have most functions from the MySQL branch implemented now as well. I'm extremely grateful to Padraig O'Sullivan for getting this project off the ground. I was a bit stuck with the new API and C++ when I first attempted these and he designed the class setup and had the major functions working which I then picked up and added more functions as well as tests. I was also glad to have the drizzle team add in my sleep() UDF which allowed me to test expirations in these memcached functions.

These functions have some similar, but very little code from the memcached Functions for MySQL. The new UDF API is completely different than MySQL's UDF API. You have to create a class for each function that is a …

Part 1 of our series set-up our "test"  application and looked at boosting performance of the application by buffer MySQL with memcached.  Our test application is simple and requires only 3 basic operations per transaction 2 reads and 1 write.  Using memcached combined with MySQL we ended up nearly getting a 10X performance boost from the application.  Now we are going to look at what we could achieve if we did not have to write to the database at all.  So let's look at what happens if we push everything including writes into memcached.

Wow that's shockingly fast isn't it! I guess being completely in memory helps for this app.  What is very interesting is accessing 100% of the data in memcached gives very similar numbers to accessing 100% of the data in memory in the DB ( part 1 …

All to often people force themselves into using a database like MySQL with no thought into whether if its the best solution to there problem. Why?  Because their other applications use it, so why not the new application?  Over the past couple of months I have been doing a ton of work for clients who use their database like most people use memcached .  Lookup a row based on a key, update the data in the row, stuff the row back in the database.  Rinse and repeat.  Sure these setups vary sometimes, throwing in a “lookup” via username, or even the rare count.  But for the most part they are designed to be simple.

A classic example is a simple online game.  An online game may only require that an application retrieve a single record from the database.  The record may contain all the vital stats for the game, be updated and stuffed back into the database.  You would be surprised how many people use …

The NoSQL event in New York had a number of presentations on non relational technologies including of Hadoop, MongoDB and CouchDB.

Coming historically from a relational background of 20 years with Ingres, Oracle and MySQL I have been moving my focus towards non relational data store. The most obvious and well used today is memcached, a non persistent distributed key/value pair store. There are a number of persistent key/value stores in the marketplace, Tokyo Cabinet, …

Key-value databases are catching fire these days. Memcached, Redis, Cassandra, Keyspace, Tokyo Tyrant, and a handful of others are surging in popularity, judging by the contents of my feed reader.

I find a number of things interesting about these tools.

• There are many more of them than open-source traditional relational databases. (edit: I mean that there are many options that all seem similar to each other, instead of 3 or 4 standing out as the giants.)
• It seems that a lot of people are simultaneously inventing solutions to their problems in private without being aware of each other, then open-sourcing the results. That points to a sudden sea change in architectures. Tipping points tend to be abrupt, which would explain isolated redundant development.
• Many of the products are feature-rich with things programmers need: diverse language bindings, APIs, embeddability, and the ability to speak familiar …

I decided to thee other day to investigate using Tokyo Tyrant because I was interested in the fact it has a memcached protocol and I wanted to get a feel for how it works with my memcached functions for MySQL (UDF). Matt Ingenthron came up with a good term recently: Mem-capable, which Tokyo Tyrant is. I find any key/value storage to be of great interest, particularly those that you simply change the port your memcached client is connecting and simply use it the same way you would memcached.

So, just what is Tokyo Tyrant?

Tokyo Tyrant is a database server, written by Mikio Hirabayashi, for Tokyo Cabinet. It provides for concurrent remote connections to Tokyo Cabinet . It provides its own binary protocol as well as a memcached and HTTP compatibility protocols. It also has C, Perl, PHP, Java, Erlang, Python and Ruby bindings. Other features Tokyo Tyrant offers:

* Hot backup and update log
* Async …

You are an Independent Software Vendor, and you want to develop, port, or test your application on Solaris or OpenSolaris? Sun's online Virtual Lab environment - EZQual - makes it easy for you, and it's free!

The Lab features pre-installed SPARC or x86 processor-based Sun servers with development tools (SunStudio and Netbeans), Java, AMP, memcached, Squid, Httplight, PostgreSQL, Solaris or OpenSolaris, and more.

In addition, thanks to the Sun's Secure Global Desktop, accessing this secure development environment over the Internet is just like running Solaris on your own laptop:

Want to know more? Check out the EZQual web page.

I have a talk scheduled tonight, for anyone who's in the Southern New Hampshire area (or greater Northern New England area): http://permalink.gmane.org/gmane.org.user-groups.linux.gnhlug.announce/743

I'll be going over a number of things, time permitting:

* memcached (in general)
* NorthScale's memcached Amazon Machine Images and Virtual Appliance
* moxi - memcached proxy
* memcached Functions for MySQL

I'll run a couple demonstrations with each, first simple memcached usage, then using two memcached instances through moxi, then I'll throw in using them with the UDFs.

I intend to see if the Boston MySQL Users Group would be interested in this talk soon as well.

Working at Northscale has been a lot of great fun lately! I have finally figured out how to get puppet to set up a stock Amazon instance with everything we need and have been impressed with how you can automate system setup with puppet. I remember when I worked at Grazr how much of a hassle it was for us to have to rebuild systems. Something like Puppet would have been a godsend.

Today I though I would post about how cool moxi is. If you don't know, moxi is a memcached proxy (http://labs.northscale.com/moxi/) which allows you to move any complexity of having to set up the list of memcached servers you are using. Also, moxi has some great features such as:

* De-duplication of requests for popular keys
* A front cache, L1 cache to avoid network hops
* Fire and forget SET (Like an async SET) this means "set a value, but don't wait to know if it …

As I outlined in Part 1 MySQL Proxy can be one tool for performing SQL analysis. The impact with any monitoring is the art of monitoring will affect the results, in this case the performance. I don’t recommend enabling this level of detailed monitoring in production, these techniques are designed for development, testing, and possibly stress testing.

This leads to the question, how do I monitor SQL in production? The simple answer to this question is, Sampling. Take a representative sample of your production system. The implementation of this depends on many factors including your programming technology stack, and your MySQL topology.

If for example you are using PHP, then defining MySQL proxy on a production system, and executing firewall rules to redirect incoming 3306 traffic to 4040 for a period of time, e.g. 2 …