Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Previous 30 Newer Entries Showing entries 91 to 120 of 181 Next 30 Older Entries

Displaying posts with tag: memcached (reset)

New Job announcment!
+0 Vote Up -0Vote Down
Some news about a change in my career path that I've been meaning to announce. I now have moved on from Lycos and now work for NorthScale Inc!

In parting Lycos, I'd like to thank them for the great challenges I had while there. I designed and developed, along with my team members in the web publishing and support from the OPs group, PHP offerings for Tripod users, a long-awaited feature that premium users can use to install numerous PHP applications. Also, with that was a great interface I developed for installing applications. It required some work on the applications themselves to make them as easy to install as possible (similar to APS). The other task while there was to switch Tripod and Angelfire blogs from Oracle to MySQL. Thank you Lycos -- It was a pleasure working with all of you!

Now, I'm excited to announce I'm working with NorthScale. This is a great team of people --



  [Read more...]
capttofu @ 2009-07-31T12:23:00
+0 Vote Up -0Vote Down
I'm pleased to announce the release of the Memcached Functions for MySQL, version 1.0.

This release contains several changes, per the ChangeLog:

1.0 Thursday, July 30, 2009 12:00:00 EST 2009
* Fixed issue of setting NULLs with user-defined variables (Thanks to
Jean-Jacques Moortgat at aol dot com !)
* Fixed issue of obtaining a NULL value FROM memcached
* All set functions now return 0 (failure) or 1 (success)
* Other cleanups
* More tests

Importantly, there was an issue that I blogged about several days ago where in the UDF API, if you pass a user-defined variable that is set to NULL to memc_set(), the length of the argument is 8192 even though the value of the argument itself is NULL, which caused much unhappiness in MySQL (crash). That is fixed by setting the length to 0 if the argument itself is NULL. Also fixed is obtaining the NULL value











  [Read more...]
WebStack 1.5 - Your (L)AMP Stack
Employee_Team +0 Vote Up -0Vote Down

Sun's LAMP support is assembled from two pieces: the L is from our Linux/GNU Support (see SunSolve entry), while the AMP comes from the GlassFish WebStack, which, in its latest incarnation includes Apache HTTP Server, lighttpd, memcached, MySQL, PHP, Python, Ruby, Squid, Tomcat, GlassFish (v2.1) and Hudson (features).

The inclusion of Hudson is a bit of an opportunistic move (more on that in a bit), the rest comprises a well tested, integrated, optimized, and extended component

  [Read more...]
451 CAOS Links 2009.07.28
+0 Vote Up -1Vote Down

Intuit launches open source project. SFLC on Microsoft GPL violation accusations.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# Intuit launched open source projects and community to develop apps based on its Intuit Partner Platform, while Savio Rodrigues declared Intuit’s open source play is all business.

# SFLC’s Bradley Kuhn told SDTimes Microsoft was in violation of the GPL.

# MySQL and Memcached-based appliance vendor Schooner Info Tech has raised $20m in Series B funding.

# Novell


  [Read more...]
memcached Functions for MySQL now on launchpad
+0 Vote Up -0Vote Down
Hi all,

This is a quick post to let you know that the memcached functions for MySQL have been moved to Launchpad. The project page is: https://launchpad.net/memcached-udfs

I think this will help to get the project more exposure, as well as making it easier for people to contribute to the project. I've found Launchpad to be quite useful for managing projects and so decided to move the UDFs there.

I'm working on getting out another version soon. I just fixed a bug the deals with user-defined variables that were set to NULL causing the UDFs to crash the server. It was a bug in the length of the argument being set to 8192


For instance, the first explicitly:

mysql> select memc_set('nullval', null);

(gdb) p args->args[1]
$3 = 0x0
(gdb) p args->lengths[1]
$4 = 0

















  [Read more...]
Analyze and optimize memcached usage with Maatkit
+5 Vote Up -0Vote Down

Ryan posted an article on the MySQL Performance Blog about how to use mk-query-digest to analyze and understand your memcached usage with the same techniques you use for MySQL query analysis. This is an idea that came to me during the 2009 MySQL Conference, while talking to our friends from Schooner, who sell a memcached appliance.

It suddenly struck me that the science of memcached performance is basically nonexistent, from the standpoint of developers and architects. Everyone treats it as a magical tool that just performs well and doesn’t need to be analyzed, which is demonstrably and self-evidently false. memcached itself is very fast, true, so it doesn’t usually become a performance bottleneck

  [Read more...]
Maatkit Now Supports Memcached
+3 Vote Up -0Vote Down

Have you ever wondered how optimized your Memcached installation is? There is a common misconception that one doesn't have to think too deeply about Memcached performance, but that is not true. If your setup is inefficient, you could:

  • Burn Memory
  • Waste Network Round-Trips
  • Store Keys That Never Get Retrieved
  • Have a Low Cache Hit Ratio (i.e. query MySQL too much)
  • Suffer a fate too horrible to contemplate.

Percona does a lot of consulting around Memcached, so we try to take a quantitative, scientific approach to measuring memcached performance, just like everything else we do.

memcached is basically a key-value in-memory database, so it works well to analyze its traffic with Maatkit's

  [Read more...]
The commercialisation of Memcached
+0 Vote Up -0Vote Down

There has been a significant increase in interest in the Memcached, the open source distributed memory object-caching system, in recent months, as a number of vendors look to exploit its popularity in Web 2.0 and social networking environments.

Like Hadoop, which has become the focus of a number of commercial plays, it would appear that the time is right for commercialization of Memcached. But what is it, here did it come from, and what are the chances for vendors to rake in serious cash? Here are the details.

What is it?
Pronounced mem-cash-dee, Memcached was originally created by Danga Interactive (the developer of LiveJournal, which was acquired by Six Apart in 2005) to speed up the performance of dynamic Web applications by alleviating database load. Memcached has become an industry standard for improving the performance of

  [Read more...]
multi-threaded memcached
+1 Vote Up -0Vote Down

I discovered while compiling Wafflegrid today that by default, the Ubuntu binaries for memcached are not-multithreaded.

Following the installation of memcached from apt-get and libmemcached I ran memslap for:

$ memslap -s localhost
    Threads connecting to servers 1
    Took 1.633 seconds to load data

$ memstat -s localhost
Listing 1 Server

Server: localhost (11211)
     pid: 23868
     uptime: 54
     time: 1244575816
     version: 1.2.2
     pointer_size: 32
     rusage_user: 0.90000
     rusage_system: 0.120000
     curr_items: 10000
     total_items: 10000
     bytes: 5430000
     curr_connections: 1
     total_connections: 3
     connection_structures: 2
     cmd_get: 0
     cmd_set: 10000
     get_hits:
  [Read more...]
OpenSolaris beats Linux on Memcached !
Employee +0 Vote Up -0Vote Down

Following on the heels of our memcached performance tests on SunFire X2270 ( Sun's Nehalem-based server) running OpenSolaris, we ran the same tests on the same server but this time on RHEL5. As mentioned in the post presenting the first memcached results, a 10GBE Intel Oplin card was used in order to achieve the high throughput rates possible with these servers. It turned out that using this card on linux involved a bit of work resulting in driver and kernel re-builds.

  • With the default ixgbe driver from the RedHat distribution (version 1.3.30-k2 on kernel 2.6.18)), the interface simply hung during the benchmark test.
  • This led to downloading the driver from the Intel site

  [Read more...]
OpenSolaris beats Linux on Memcached !
Employee +0 Vote Up -0Vote Down

Following on the heels of our memcached performance tests on SunFire X2270 ( Sun's Nehalem-based server) running OpenSolaris, we ran the same tests on the same server but this time on RHEL5. As mentioned in the post presenting the first memcached results, a 10GBE Intel Oplin card was used in order to achieve the high throughput rates possible with these servers. It turned out that using this card on linux involved a bit of work resulting in driver and kernel re-builds.

  • With the default ixgbe driver from the RedHat distribution (version 1.3.30-k2 on kernel 2.6.18)), the interface simply hung during the benchmark test.
  • This led to downloading the driver from the Intel site

  [Read more...]
OpenSolaris beats Linux on Memcached !
Employee +0 Vote Up -0Vote Down

Following on the heels of our memcached performance tests on SunFire X2270 ( Sun's Nehalem-based server) running OpenSolaris, we ran the same tests on the same server but this time on RHEL5. As mentioned in the post presenting the first memcached results, a 10GBE Intel Oplin card was used in order to achieve the high throughput rates possible with these servers. It turned out that using this card on linux involved a bit of work resulting in driver and kernel re-builds.

  • With the default ixgbe driver from the RedHat distribution (version 1.3.30-k2 on kernel 2.6.18)), the interface simply hung during the benchmark test.
  • This led to downloading the driver from the Intel site

  [Read more...]
Scaling Memcached: 500,000+ Operations/Second with a Single-Socket UltraSPARC T2
Employee +0 Vote Up -0Vote Down

A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns.

Table below shows out-of-the-box (no custom OS rewrites or networking tuning required) performance with 10G networking hardware and one single-socket UltraSPARC T2-based server with 8 cores and 8 threads per core (64 threads on a chip). All runs are done with a single memcached instance and 40 worker threads so that about 3 cores (24 threads) are used for the critical networking stack that is also heavily

  [Read more...]
Scaling Memcached: 500,000+ Operations/Second with a Single-Socket UltraSPARC T2
Employee +0 Vote Up -0Vote Down

A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns.

Table below shows out-of-the-box (no custom OS rewrites or networking tuning required) performance with 10G networking hardware and one single-socket UltraSPARC T2-based server with 8 cores and 8 threads per core (64 threads on a chip). All runs are done with a single memcached instance and 40 worker threads so that about 3 cores (24 threads) are used for the critical networking stack that is also heavily

  [Read more...]
Connection pooling libmemcached
+0 Vote Up -0Vote Down

A while back I looked at the Memcached UDF for MySQL, and noticed that it didn't use libmemcached in an optimal way. In order to work in a multithreaded environment it used the following pattern:

   memcached_st* clone = memcached_clone(NULL, memc);

   ... memcached operations using the clone ---   

   memcached_free(clone);

Well, that doesn't look bad, does it? Well, it isn't that bad, but if you look at the network traffic you will see that we end up connecting / disconnecting to the involved memcached servers every time, and memcached is not optimized for "single-shot" connections.

So how should you solve this? Well, you should reuse your clones! And luckily for you, you don't have to reinvent the wheel. Yesterday I pushed a patch to libmemcached introducing a new library: libmemcachedutil. The intention of that

  [Read more...]
MySQL Proxy - proxydb
Employee +0 Vote Up -0Vote Down
What I really like about having Lua and MySQL Proxy together is that it turns out to be very flexible, you can have the proxy do all kinds of things. And the last thing I made the MySQL Proxy do is to act like memcached.

Well, maybe not, but it handles key => value pairs now :P




What does it do?

It handles 5 basic query types:

mysql> INSERT "key" = "value";

Very simple, insert a key => value pair on a proxy.global.db table, if there is already a value for that query, it will overwrite it.

mysql> SELECT "key";














  [Read more...]
Version 1.1.2 of improved Cacti templates released
+0 Vote Up -0Vote Down

I’ve packaged up and released version 1.1.2 of the Cacti templates I’ve written for MySQL, Apache, memcached, nginx etc.

Anyone who would like to help write documentation (or do anything else, for that matter) is welcomed to participate. I’ll give commit access at the drop of a hat.

Changelog:

2009-05-07: version 1.1.2

	* The parsing code did not handle InnoDB plugin / XtraDB (issue 52).
	* The servername was hardcoded in ss_get_by_ssh.php (issue 57).
	* Added Handler_ graphs (issue 47).
	* Config files can be used instead of editing the .php file (issue 39).
	* binary log space is now calculated without a MySQL query (issue 48).
	* There was no easy way to force inputs to be filled (issue 45).
	* Some graphs were partially hidden without
  [Read more...]
Multi-instance memcached performance
Employee +0 Vote Up -0Vote Down

As promised, here are more results running memcached on Sun's X2270 (Nehalem-based server). In my previous post, I mentioned that we got 350K ops/sec running a single instance of memcached at which point the throughput was hampered by the scalability issues of memcached. So we ran two instances of memcached on the same server, each using 15GB of memory and tested both 1.2.5 and 1.3.2 versions. Here are the results :

The maximum throughput was 470K ops/sec using 4 threads in memcached 1.3.2. Performance of 1.2.5 was just very slightly lower. At this throughput, the network capacity of the single 10gbe card was reached as the benchmark does a lot of small packet transfers. See my

  [Read more...]
Multi-instance memcached performance
Employee +0 Vote Up -0Vote Down

As promised, here are more results running memcached on Sun's X2270 (Nehalem-based server). In my previous post, I mentioned that we got 350K ops/sec running a single instance of memcached at which point the throughput was hampered by the scalability issues of memcached. So we ran two instances of memcached on the same server, each using 15GB of memory and tested both 1.2.5 and 1.3.2 versions. Here are the results :

The maximum throughput was 470K ops/sec using 4 threads in memcached 1.3.2. Performance of 1.2.5 was just very slightly lower. At this throughput, the network capacity of the single 10gbe card was reached as the benchmark does a lot of small packet transfers. See my

  [Read more...]
Multi-instance memcached performance
Employee +0 Vote Up -0Vote Down

As promised, here are more results running memcached on Sun's X2270 (Nehalem-based server). In my previous post, I mentioned that we got 350K ops/sec running a single instance of memcached at which point the throughput was hampered by the scalability issues of memcached. So we ran two instances of memcached on the same server, each using 15GB of memory and tested both 1.2.5 and 1.3.2 versions. Here are the results :

The maximum throughput was 470K ops/sec using 4 threads in memcached 1.3.2. Performance of 1.2.5 was just very slightly lower. At this throughput, the network capacity of the single 10gbe card was reached as the benchmark does a lot of small packet transfers. See my

  [Read more...]
Secure, easy Cacti graphing without SNMP
+1 Vote Up -0Vote Down

Cacti is a great tool for collecting information about systems and graphing it. However, it likes to use SNMP, and SNMP is often not desirable. Instead, I often see the need for a method that is:

  • Secure. Use trusted, well-known, encrypted communication. Do not open up new ports.
  • Zero install on the monitored system.
  • As little installation or modification on the monitoring system as possible.

Over the last several years, I’ve slowly created more and more software to create Cacti graphs via standard POSIX command-line utilities over SSH with key-pair authentication. (I’ve also created similar software for Nagios, but that’s another matter.) The major problem with the work I’ve done is that it’s totally un-publicized.

The system works by

  [Read more...]
Presentation at the MySQL Users Conference
+0 Vote Up -0Vote Down

Earlier today I did the presentation Memcached Meet Flash, the pluggable engine interface, and if you missed it you can download the slides. It is kind of fun to think back on the hackathon at the users conference the last year when Toru shared his ideas about a storage interface, followed by the interesting discussion I had with Matt during the OpenSolaris summit down in Santa Clara. I didn't know back then that I would present this at the users conference this year :-)

My brother came down for my presentation and took the following picture with his iPhone during the session:

  [Read more...]
Memcached Performance on Sun's Nehalem System
Employee +0 Vote Up -0Vote Down

Memcached is the de-facto distributed caching server used to scale many web2.0 sites today. With the requirement to support a very large number of users as sites grow, memcached aids scalability by effectively cutting down on MySQL traffic and improving response times.

Memcached is a very light-weight server but is known not to scale beyond 4-6 threads. Some scalability improvements have gone into the 1.3 release (still in beta). With the new Intel Nehalem based systems improved hyper-threading providing twice as much performance as current systems, we were curious to see how memcached would perform on these systems. So we ran some tests, the results of which are shown below :





  [Read more...]
Memcached Performance on Sun's Nehalem System
Employee +0 Vote Up -0Vote Down

Memcached is the de-facto distributed caching server used to scale many web2.0 sites today. With the requirement to support a very large number of users as sites grow, memcached aids scalability by effectively cutting down on MySQL traffic and improving response times.

Memcached is a very light-weight server but is known not to scale beyond 4-6 threads. Some scalability improvements have gone into the 1.3 release (still in beta). With the new Intel Nehalem based systems improved hyper-threading providing twice as much performance as current systems, we were curious to see how memcached would perform on these systems. So we ran some tests, the results of which are shown below :





  [Read more...]
Memcached Performance on Sun's Nehalem System
Employee +0 Vote Up -0Vote Down

Memcached is the de-facto distributed caching server used to scale many web2.0 sites today. With the requirement to support a very large number of users as sites grow, memcached aids scalability by effectively cutting down on MySQL traffic and improving response times.

Memcached is a very light-weight server but is known not to scale beyond 4-6 threads. Some scalability improvements have gone into the 1.3 release (still in beta). With the new Intel Nehalem based systems improved hyper-threading providing twice as much performance as current systems, we were curious to see how memcached would perform on these systems. So we ran some tests, the results of which are shown below :





  [Read more...]
CAOS Theory Podcast 2009.04.17
+0 Vote Up -0Vote Down

Topics for this podcast:

*CAOS 11 - Open to Investment
*CollabNet out with new TeamForge 5.2
*Memcached and MySQL appliances abound

iTunes or direct download (25:05, 5.8 MB)

Frank Mashraqi on Hadoop, memcached, and why the MySQL Conference is cool
+0 Vote Up -0Vote Down

Today I spoke with Farhan “Frank” Mashraqi, former Fotolog DBA, now working at a startup, NetEdge, working on social analytics. He’s talking about the two sessions he’s giving next week at the MySQL Conference & Expo 2009, as well as the benefits of being at the MySQL Conference & Expo.



He’s giving two talks:

  • Hadoop and MySQL: Friends with Benefits in where he will tell you about how you can combine data sets and queries, some of which run on Hadoop, and others which run on MySQL, but eventually probably end up in MySQL (he


  •   [Read more...]
    Memcached Functions For MySQL 0.9 Released
    +0 Vote Up -0Vote Down
    I'm glad to announce the release of Memcached Functions for MySQL version 0.9. This version contains a couple bug fixes:

    * Switched to using calloc() instead of malloc(), check for return value
    * Changed memc_servers_set() to not create a new connection when one already exists, free first

    Also added:

    * memc_server_version()

    mysql> select memc_server_version('127.0.0.1:11211')\G
    *************************** 1. row ***************************
    memc_server_version('127.0.0.1:11211'): Server: 127.0.0.1 (11211) version 1.2.6

    I'm glad to get development moving along again. The book really took a ton of time, so now I can get this project moving along. Version 0.10 I'll be adding even more functionality. I'll get some ideas at the user's conference next week as well. One thing I need to get working properly is the ability to set a value in a












      [Read more...]
    Memcached 1.2.7 and 1.3.3
    +0 Vote Up -0Vote Down
    original post

    ... and this is my usual plea to those mysql/web/industrial folks to try out the latest code. Help us on our quest to scale the crap out of all of your stuff. :)

    Find us on the mailing list, on #memcached on freenode, or on twitter as dormando, dlsspy, tmaesaka, and trondn. :) All others are fakers.

    ---

    Two new memcached releases are available today.

    Stable 1.2.7

    The new stable release which is a maintenance release of the 1.2
    series containing several bugfixes and a few features.

    This version














      [Read more...]
    What is the official branch of MySQL?
    +0 Vote Up -0Vote Down
    This week, I'm finishing up my book "Developing Web Applications using Perl, memcached, MySQL and Apache". I just finished up Chapter 1, which is the first chapter where I discuss several things such as how much things have changed in the last ten years with web development and Open Source in general. This lead me to write a small section about the future. I was pondering -- where are we going to be in ten years from now? What projects will be popular? This lead me even to a more specific question, and one that I would ask here: what is to become with the development of MySQL? What will be the official development branch of MySQL, and where is all the energy and excitement going to be around?

    There is now one fork and one major branch of MySQL now that I will mention, both are exciting projects-- and neither of them is coming from official MySQL/Sun:

    (The order of these is strictly



      [Read more...]
    Previous 30 Newer Entries Showing entries 91 to 120 of 181 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.