Within the ideeli application we have a subset of requests which rely on data that is very expensive to generate into a format optimized for the frontend of the site. There are a number of strategies to make this more performant which we will discuss in future posts; for now let’s just assume that we have a json document of sku availability data which takes 30 seconds to compute (from thousands of mysql queries and some processing in the ruby domain).

This data within this json document is time-sensitive (e.g. whether or not the Add-to-cart buttons should show as sold out). So whilst we are happy to cache it for a few minutes, we have to regenerate it on a frequent basis. The typical solution with rails would be to throw an action cache against the particular url, with a 5 minute expiry.

We have a problem with this. When the page expires, all of the requests to the url - for the next 30 seconds that it takes to generate the page …

Continuent today announced packaged open source support for real-time replication from MySQL to Oracle using Tungsten Replicator, Continuent’s award-winning replication software. Customers can now sign up for a cost-effective engagement from a Continuent replication expert to help install, configure and test replication from MySQL to Oracle using Tungsten Replicator. Customers can also add

Cloudera raises $40m. Accel announces $100m fund. Rackspace takes OpenStack private. And more.

# Cloudera raised $40m in series D funding and announced a partnership with NetApp around its NetApp Open Solution for Hadoop.

# Accel Partners launched a $100m Big Data Fund to invest in Hadoop- and NoSQL-related vendors.

# Rackspace …

Opening a database connection is a boring tasks. But do you know how defaults are determined, if values are omitted? Or, did you know there are two flavours of persistent connections in mysqli? Of course you, as a german reader, know it. I blogged about it in 2009 over at phphatesme.com (Nimmer Ärger mit den Persistenten Verbindungen von MySQL? ) …

Database connections with mysqli

The MySQL server supports the use of different transport layers for connections. Connections use TCP/IP, Unix domain sockets or Windows named pipes.

The hostname localhost has a special meaning. It is bound to the use of Unix domain sockets. It is not possible to open a TCP/IP connection using the hostname localhost you must use 127.0.0.1 instead.

$mysqli = new mysqli("localhost", "root", "", "test");
echo …

One of the things I think we did right with Maatkit (and now with Percona Toolkit) is making the documentation part of the code itself. So much redundancy and wrong documentation has been eliminated by making the tool actually read its own documentation when it starts up. As an example, the default value of the –shorten option is defined in the documentation (it’s Perldoc) like this:

=item --shorten

type: int; default: 1024

Not only is the documentation part of the code, but the tool’s –help output is generated from it too. The existence, type, defaults, and even the behavior of the command-line options is defined in the documentation. If I execute the tool with the –help option, you can see that default value:

[baron@ginger bin]$ ./pt-query-digest --help | grep  -- --shorten
  --shorten=i                    Shorten long statements in reports (default
  --shorten                      1024 …

Oracle recently provided access to many new MySQL 5.6 features through http://labs.mysql.com for the user community to test and comment on. One notable feature is the InnoDB Full-Text Search (FTS) engine. It lets users build FULLTEXT indexes on InnoDB tables to represent text-based content and speed up searches for words and phrases.

We had a lot of great technical presentations at Percona Live, London If you did not have a chance to attend or if you did but could not be at 5 rooms at the same time you can check out slides which have been just made available for download for most of the talks. Enjoy and see you on our future events !

So I’m back from the Percona dev team’s recent meeting.  While there, we spent a fair bit of time discussing Xtrabackup development.  One of our challenges is that as we add richer features to the tool, we need equivalent testing capabilities.  However, it seems a constant in the MySQL world that available QA tools often leave something to be desired.  The randgen is a literal wonder-tool for database testing, but it is also occasionally frustrating / doesn’t scratch every testing itch.  It is based on technology SQL Server was using in 1998 (MySQL began using it in ~2007, IIRC).  So this is no knock, it is merely meant to be an example of a poor QA engineer’s frustrations ; )  While the current …

MySQL Enterprise Monitor or MEM is a tool to watch over one instance to a farm of MySQL servers, to warn you of problems, and can advise you on fixing problems.

The is the MySQL Enterprise Monitor Dashboard

But what does it take it get it running? How much can it show me about my server?

Start by downloading MEM from the Oracle Software Delivery Cloud (And it is still free for a 30 day trial). In addition to the monitor, make sure you download the monitor agent. For my 32-bit Ubuntu test box, there were named mysqlmonitor-2.3.7.2104-linux-x86-installer.bin and mysqlmonitoragent-2.3.7.2104-linux-x86-installer.bin.

Executing …

Due to the insane cost of bandwidth (compared to the rest of the developed world) in Australia, I've recently decided to move some of our hosting clients to Linode. This means they can move more data more cheaply and I don't need to come up with (and administer) a bandwidth accounting system for my Australian based web VM.

We pretty much exclusively use Drupal for hosting clients, so to make management a bit easier I decided to use Ægir on the new Linode. Installation was a relative breeze, after a quick google to find out how to specify that I didn't want to use Apache and wanted to use a separate server as dedicated MySQL host.

The problem (there is always a problem) arose when I needed to give a hosting client access to their Drupal installation, so they could manage themes …