| Previous 30 Newer Entries | Showing entries 91 to 120 of 120 |
Ryan posted an article on the MySQL Performance Blog about how to use mk-query-digest to analyze and understand your memcached usage with the same techniques you use for MySQL query analysis. This is an idea that came to me during the 2009 MySQL Conference, while talking to our friends from Schooner, who sell a memcached appliance.
It suddenly struck me that the science of memcached performance is basically nonexistent, from the standpoint of developers and architects. Everyone treats it as a magical tool that just performs well and doesn’t need to be analyzed, which is demonstrably and self-evidently false. memcached itself is very fast, true, so it doesn’t usually become a performance bottleneck
[Read more...]Have you ever wondered how optimized your Memcached installation is? There is a common misconception that one doesn't have to think too deeply about Memcached performance, but that is not true. If your setup is inefficient, you could:
Percona does a lot of consulting around Memcached, so we try to take a quantitative, scientific approach to measuring memcached performance, just like everything else we do.
memcached is basically a key-value in-memory database, so it works well to analyze its traffic with Maatkit's
[Read more...]There has been a significant increase in interest in the Memcached, the open source distributed memory object-caching system, in recent months, as a number of vendors look to exploit its popularity in Web 2.0 and social networking environments.
Like Hadoop, which has become the focus of a number of commercial plays, it would appear that the time is right for commercialization of Memcached. But what is it, here did it come from, and what are the chances for vendors to rake in serious cash? Here are the details.
What is it?
Pronounced mem-cash-dee, Memcached was originally created by Danga Interactive (the developer of LiveJournal, which was acquired by Six Apart in 2005) to speed up the performance of dynamic Web applications by alleviating database load. Memcached has become an industry standard for improving the performance of
I discovered while compiling Wafflegrid today that by default, the Ubuntu binaries for memcached are not-multithreaded.
Following the installation of memcached from apt-get and libmemcached I ran memslap for:
$ memslap -s localhost
Threads connecting to servers 1
Took 1.633 seconds to load data
$ memstat -s localhost
Listing 1 Server
Server: localhost (11211)
pid: 23868
uptime: 54
time: 1244575816
version: 1.2.2
pointer_size: 32
rusage_user: 0.90000
rusage_system: 0.120000
curr_items: 10000
total_items: 10000
bytes: 5430000
curr_connections: 1
total_connections: 3
connection_structures: 2
cmd_get: 0
cmd_set: 10000
get_hits: [Read more...]
Following on the heels of our memcached performance tests
on SunFire X2270 ( Sun's Nehalem-based server) running OpenSolaris, we
ran the same tests on the same server but this time on RHEL5. As
mentioned in the post presenting the first memcached results,
a 10GBE Intel Oplin card was used in order to achieve the high
throughput rates possible with these servers. It turned out that using
this card on linux involved a bit of work resulting in driver and kernel
re-builds.
Following on the heels of our memcached performance tests
on SunFire X2270 ( Sun's Nehalem-based server) running OpenSolaris, we
ran the same tests on the same server but this time on RHEL5. As
mentioned in the post presenting the first memcached results,
a 10GBE Intel Oplin card was used in order to achieve the high
throughput rates possible with these servers. It turned out that using
this card on linux involved a bit of work resulting in driver and kernel
re-builds.
Following on the heels of our memcached performance tests
on SunFire X2270 ( Sun's Nehalem-based server) running OpenSolaris, we
ran the same tests on the same server but this time on RHEL5. As
mentioned in the post presenting the first memcached results,
a 10GBE Intel Oplin card was used in order to achieve the high
throughput rates possible with these servers. It turned out that using
this card on linux involved a bit of work resulting in driver and kernel
re-builds.
A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns.
Table below shows out-of-the-box (no custom OS rewrites or networking tuning required) performance with 10G networking hardware and one single-socket UltraSPARC T2-based server with 8 cores and 8 threads per core (64 threads on a chip). All runs are done with a single memcached instance and 40 worker threads so that about 3 cores (24 threads) are used for the critical networking stack that is also heavily
[Read more...]A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns.
Table below shows out-of-the-box (no custom OS rewrites or networking tuning required) performance with 10G networking hardware and one single-socket UltraSPARC T2-based server with 8 cores and 8 threads per core (64 threads on a chip). All runs are done with a single memcached instance and 40 worker threads so that about 3 cores (24 threads) are used for the critical networking stack that is also heavily
[Read more...]A while back I looked at the Memcached UDF for MySQL, and noticed that it didn't use libmemcached in an optimal way. In order to work in a multithreaded environment it used the following pattern:
memcached_st* clone = memcached_clone(NULL, memc); ... memcached operations using the clone --- memcached_free(clone);
Well, that doesn't look bad, does it? Well, it isn't that bad, but if you look at the network traffic you will see that we end up connecting / disconnecting to the involved memcached servers every time, and memcached is not optimized for "single-shot" connections.
So how should you solve this? Well, you should reuse your clones! And luckily for you, you don't have to reinvent the wheel. Yesterday I pushed a patch to libmemcached introducing a new library: libmemcachedutil. The intention of that

mysql> INSERT "key" = "value";
proxy.global.db table, if there is already a value for that query, it will overwrite it.[Read more...]
mysql> SELECT "key";
I’ve packaged up and released version 1.1.2 of the Cacti templates I’ve written for MySQL, Apache, memcached, nginx etc.
Anyone who would like to help write documentation (or do anything else, for that matter) is welcomed to participate. I’ll give commit access at the drop of a hat.
Changelog:
2009-05-07: version 1.1.2 * The parsing code did not handle InnoDB plugin / XtraDB (issue 52). * The servername was hardcoded in ss_get_by_ssh.php (issue 57). * Added Handler_ graphs (issue 47). * Config files can be used instead of editing the .php file (issue 39). * binary log space is now calculated without a MySQL query (issue 48). * There was no easy way to force inputs to be filled (issue 45). * Some graphs were partially hidden without[Read more...]
As promised, here are more results running memcached on Sun's X2270 (Nehalem-based server). In my previous post, I mentioned that we got 350K ops/sec running a single instance of memcached at which point the throughput was hampered by the scalability issues of memcached. So we ran two instances of memcached on the same server, each using 15GB of memory and tested both 1.2.5 and 1.3.2 versions. Here are the results :

The maximum throughput was 470K ops/sec using 4 threads in memcached 1.3.2. Performance of 1.2.5 was just very slightly lower. At this throughput, the network capacity of the single 10gbe card was reached as the benchmark does a lot of small packet transfers. See my
[Read more...]As promised, here are more results running memcached on Sun's X2270 (Nehalem-based server). In my previous post, I mentioned that we got 350K ops/sec running a single instance of memcached at which point the throughput was hampered by the scalability issues of memcached. So we ran two instances of memcached on the same server, each using 15GB of memory and tested both 1.2.5 and 1.3.2 versions. Here are the results :

The maximum throughput was 470K ops/sec using 4 threads in memcached 1.3.2. Performance of 1.2.5 was just very slightly lower. At this throughput, the network capacity of the single 10gbe card was reached as the benchmark does a lot of small packet transfers. See my
[Read more...]As promised, here are more results running memcached on Sun's X2270 (Nehalem-based server). In my previous post, I mentioned that we got 350K ops/sec running a single instance of memcached at which point the throughput was hampered by the scalability issues of memcached. So we ran two instances of memcached on the same server, each using 15GB of memory and tested both 1.2.5 and 1.3.2 versions. Here are the results :

The maximum throughput was 470K ops/sec using 4 threads in memcached 1.3.2. Performance of 1.2.5 was just very slightly lower. At this throughput, the network capacity of the single 10gbe card was reached as the benchmark does a lot of small packet transfers. See my
[Read more...]Cacti is a great tool for collecting information about systems and graphing it. However, it likes to use SNMP, and SNMP is often not desirable. Instead, I often see the need for a method that is:
Over the last several years, I’ve slowly created more and more software to create Cacti graphs via standard POSIX command-line utilities over SSH with key-pair authentication. (I’ve also created similar software for Nagios, but that’s another matter.) The major problem with the work I’ve done is that it’s totally un-publicized.
The system works by
[Read more...]Earlier today I did the presentation Memcached Meet Flash, the pluggable engine interface, and if you missed it you can download the slides. It is kind of fun to think back on the hackathon at the users conference the last year when Toru shared his ideas about a storage interface, followed by the interesting discussion I had with Matt during the OpenSolaris summit down in Santa Clara. I didn't know back then that I would present this at the users conference this year :-)
My brother came down for my presentation and took the following picture with his iPhone during the session:
[Read more...]Memcached
is the de-facto distributed caching server used to scale many web2.0
sites today. With the requirement to support a very large number of
users as sites grow, memcached aids scalability by effectively cutting
down on MySQL traffic and improving response times.
Memcached
is a very light-weight server but is known not to scale beyond 4-6
threads. Some scalability improvements have gone into the 1.3 release
(still in beta). With the new Intel Nehalem based systems improved
hyper-threading providing twice as much performance as current systems,
we were curious to see how memcached would perform on these systems. So we ran some tests, the results of which are shown below :

Memcached
is the de-facto distributed caching server used to scale many web2.0
sites today. With the requirement to support a very large number of
users as sites grow, memcached aids scalability by effectively cutting
down on MySQL traffic and improving response times.
Memcached
is a very light-weight server but is known not to scale beyond 4-6
threads. Some scalability improvements have gone into the 1.3 release
(still in beta). With the new Intel Nehalem based systems improved
hyper-threading providing twice as much performance as current systems,
we were curious to see how memcached would perform on these systems. So we ran some tests, the results of which are shown below :
Memcached
is the de-facto distributed caching server used to scale many web2.0
sites today. With the requirement to support a very large number of
users as sites grow, memcached aids scalability by effectively cutting
down on MySQL traffic and improving response times.
Memcached
is a very light-weight server but is known not to scale beyond 4-6
threads. Some scalability improvements have gone into the 1.3 release
(still in beta). With the new Intel Nehalem based systems improved
hyper-threading providing twice as much performance as current systems,
we were curious to see how memcached would perform on these systems. So we ran some tests, the results of which are shown below :
Topics for this podcast:
*CAOS 11 - Open to Investment
*CollabNet out with new TeamForge 5.2
*Memcached and MySQL appliances abound
iTunes or direct download (25:05, 5.8 MB)
Today I spoke with Farhan “Frank” Mashraqi, former Fotolog DBA, now working at a startup, NetEdge, working on social analytics. He’s talking about the two sessions he’s giving next week at the MySQL Conference & Expo 2009, as well as the benefits of being at the MySQL Conference & Expo.
He’s giving two talks:
I got an email yesterday about a user experiencing problems when using memcached UDF for MySQL, so today I spent some time trying to recreate the problem. It turned out that the bug was caused by using uninitialized memory, so I guessed it could be a good blog documenting how I found it...
The first thing I did was to compile and install libmemcached and the udf:
trond@opensolaris> ./config/bootstrap trond@opensolaris> ./configure --prefix=/opt/memcached --enable-dependency-tracking --enable-debug --without-memcached trond@opensolaris> gmake all install trond@opensolaris> ./config/bootstrap trond@opensolaris> ./configure --prefix=/opt/memcached --with-mysql=/usr/mysql/bin/mysql_config --with-libmemcached=/opt/memcached CFLAGS=-g trond@opensolaris> gmake all install
The next thing to do is to
[Read more...]
| Previous 30 Newer Entries | Showing entries 91 to 120 of 120 |