You might have seen my attempts at testing MySQL as a Key Value Store (KVS) (here and here), i.e. used like MongoDB, and I have had some comments to those. Above all, many wanted to know the details of my test environment. To be honest, this started off as a very simple thing, but it turned out a bigger than I thought, so I guess its reasonable to give some background. I apologize for not giving you this background before, but as I said, my initial intention was to just run bog standard MongoDB versus MySQL and see what happened.  [Read more...]

1. Percona Toolkit
Percona Toolkit (aka Maatkit and Aspersa) is must have collection of advanced command-line tools which helps in performing tasks that are too difficult or complex to perform manually.

2. Mydumper
Mydumper is a high-performance multi-threaded backup/restore tool for MySQL. It’s up to 10x faster compared to mysqldump, can take consistent snapshots and provides File compression on-the-fly. Though it’s still under active development but is well tested/used in production on some large installations.

3. MySQL Master HA
This tool helps to maintain your Master-Slave

  [Read more...]

Read the original article at $1000 per hour Servers, Anyone?

Amazon's spot market for computing power is set up as an open market for surplus servers. The price is dynamic and depends on demand. So when demand is low, you can get computing instances for rock bottom prices. When you do that you normally set a range of prices you're willing to pay. If it goes over your top end, your instances get killed and re-provisioned for someone else. Obviously this wouldn't work for all applications, like a website that has to be up all the time, but for computing power, say to run some huge hedge fund analytics, it might fit perfectly.

A recent post on SEO MOZ alerted us to an interesting story where spot instances spiked to

  [Read more...]

I had earlier written a post on tuning the MySQL server configuration which was more geared towards the MyISAM storage engine. While that is not because I didn't intend on ignoring InnoDB but because I had planned a whole post on tuning InnoDB related configuration. So this post is the post that I had planned, I have discussed the major configuration parameters in here that should help you out most of the times.

MySQL 5.5 has created a lot of hype and its not just hype, there are major performance enhancements not only in the MySQL server itself but in the newer InnoDB plugin shipped with MySQL 5.5. That's exactly the reason why I have myself upgraded to MySQL 5.5 (The server running this blog run MySQL 5.5). Now since I haven't come across a guide to help in upgrading to MySQL 5.5, I thought why not make one myself

A couple of question I get a lot from MySQL customers is “how will this hardware upgrade improve my transactions per second (TPS)” and “what level of TPS will MySQL perform on this hardware if I’m running ACID settings?” Running sysbench against MySQL with different values for per-thread and global memory buffer sizes, ACID settings, and other settings gives me concrete values to bring to the customer to show the impact that more RAM, faster CPUs, faster disks, or cnf changes have on the server. Here are some examples for a common question: “If I’m using full ACID settings vs non-ACID settings what performance am I going to get from this server?”

Let’s find out by running sysbench with the following settings (most are self explanatory – if not the man page can explain them):

• sysbench –test=oltp

  [Read more...]

If you’ve been reading up on the various NoSQL offerings and have wanted to try out one but don’t know how to get started, this is one of the easiest ways. I chose MongoDB for this example because I’m going to start using it for a project that needs features that MySQL isn’t as fast at: namely denormalized data with billions of rows. MongoDB has plenty of drivers for other scripting and high-level languages but I’ll focus on the PHP driver today. If there is interest I can do a write up on Python usage later. This example is limited to CentOS, Fedora, and Redhat 5 servers that use the yum package management system. For more information you can reference their download page: http://www.mongodb.org/display/DOCS/Downloads

First install the prerequisites:

• sudo yum

  [Read more...]

Saw this interesting paper about highly concurrent programming methods and figured the word should be spread! It’s not new material but it’s a good read. See the full article here: http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/

“Highly concurrent applications such as Internet servers and transaction processing databases present a number of challenges to application designers. First, handling large numbers of concurrent tasks requires the use of scalable data structures. Second, these systems typically operate near maximum capacity, which creates resource contention and high sensitivity to scheduling decisions; overload must be handled with care to avoid thrashing. Finally, race conditions and subtle corner cases are common, which makes

  [Read more...]

This is a small revision and will only be available through SVN. However, it is an important one to speak about as it solves a former issue when running the application on a screen smaller than 1024px wide. While most users may not have noticed this since they have larger monitors it has been noticed [...]

When I saw Shlomi's post on why not to use apt-get or yum for MySQL, I thought immediately that his conclusions are quite reasonable. What you get from the Linux distributions is not the same thing that you find in the official MySQL downloads page. Now, whether you value more the completeness of the server or the ease of administration through the distribution installation tools, it's up to you and your business goals. We at the MySQL team have organized a meeting with the Linux distributions with the intent of finding out which differences and problems we may have with each other, and to solve them by improving communication. What follows is a summary of what happened in Brussels during the meeting.



  [Read more...]

It didn’t take long, but the advanced analytics reporting code has been added to the command line reporter version of Kontrollbase. Click here to see an example of the tuning and analytics report output (html format). Now you have all of the reporting features (minus the growth over time) features that are offered in the [...]

In my last post, I mentioned the factors that affect web performance. Now that we know what we need to measure, we come to the harder problem of figuring out how to measure each of them. There are different methods depending on how much control you have over the system and the environment it runs in. Additionally, measuring performance in a test setup may not show you what real users experience, however it does give you a good baseline to compare subsequent tests against.

Web, application and database servers

Back end servers are the easiest to measure because we generally have full control over the system and the environment it runs in. The set up is also largely the same in a test and production environment, and by replaying HTTP logs, it's possible to simulate real user interactions with the
  [Read more...]

In the past few weeks I've been implementing advanced search at Plaxo, working quite closely with Solr enterprise search server. Today, I saw this relatively detailed comparison between Solr and its main competitor Sphinx (full credit goes to StackOverflow user mausch who had been using Solr for the past 2 years). For those still confused, Solr and Sphinx are similar to MySQL FULLTEXT search, or for those even more confused, think Google (yeah, this is a bit of a stretch, I know).

Similarities

• Both Solr and Sphinx satisfy all of your requirements. They're fast and designed to index and search large bodies of data efficiently.
• Both

  [Read more...]

An example of a basic MSSQL (Microsoft SQL Server/SQL Server Express) query using PHP.

1
2
3
4
5
6
7
8
9

$szQry = "SELECT column1, column2 FROM foo";
$szDBConn = mssql_connect("host","username","password");
mssql_select_db("database_name", $szDBConn);
$saResults = mssql_query($szQry, $szDBConn);
while($obResults = mssql_fetch_row($saResults))
{
   echo $obResults[0]." ".$obResults[1];
}
mssql_close($szDBConn);

Comments/description of Example

Line #1 SQL statement that will be sent to the MySQL database server. Line #2 MSSQL database login credentilas; host (127.0.0.1), username and password. The “host” is the server name or IP address of your database server. If your host has multiple  [Read more...]

This video has been going around for some time now, but I think its worth the post for those that have yet to see it.

Video Source: The Web Site is Down

It’s been a few months since we‘ve started actively using ActiveMQ queue server in our project. For some time we had pretty weird problems with it and even started thinking about switching to something else or even writing our own queue server which would comply with our requirements. The most annoying problem was the following: some time after activemq restart everything worked really well and then activemq started lagging, queue started growing and all producer processes were stalling on push() operations. We rewrote our producers from Ruby to JRuby, then to Java and still – after some time everything was in a bad shape until we restarted the queue server.

So, long story short, after a lots of docs and source code reading we’ve found really interesting thing. There is a

  [Read more...]

Here is my experience trying to run OpenBSD with XenServer 5 Enterprise.

• XenServer console doesn’t function properly as it keeps overlaying text displayed previously or anything you have typed into the console. Makes it very difficult to read and see what you are doing. As well it appears numerpad with numlock on does not work either. The best work around is to SSH into OpenBSD.
• Receiving the following error messages at boot up, “clock: unknown CMOS layout” and “rl0: watchdog timeout”. Yes the NIC is being detected as a Realteak 8139. If I check /var/run/dmesg.boot out of the two error messages I only see the “clock: unknown CMOS layout”. So I would assume the watchdog timeout

  [Read more...]

How would one concatenate strings from a column (multiple rows) into a single row using MySQL? I see its possible with MS SQL Server 2005 and above. Any incite into how to achieve this in MySQL would be much appreciated.

MS SQL Server 2005 - Example

---

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

SELECT Web_Account_ID,
GroupNameConcat = REPLACE(
	(
	SELECT
		Web_Account_Group_Name_ID AS [DATA()]
	FROM
		tblWebAccountGroup WAG
	WHERE
		WAG.Web_Account_ID = WA.Web_Account_ID
	ORDER BY
		Web_Account_Group_Name_ID
            FOR XML PATH ('')
        ), ' ', ',')
FROM tblWebAccounts WA
ORDER BY Web_Account_ID

Query Results Example

---

  [Read more...]

Wouldn't it be cool to enhance MySQL creation table syntax to do this?

CREATE TABLE t1 (
  my_date DATE,
  my_month VIRTUAL INT AS (DATE_FORMAT(d,'%Y%m'))
);

And later, when you insert '2008-08-23' in my_date, then automatically my_month will be set to 200808.
Yes, it would be cool, and it has been done. Andrey Zhakov, a so far unknown to me community contributor, has extended the server code to implement virtual columns (WL#411) and functional indexes (





  [Read more...]

I want to get opinions from outside of my daily circle of people on the best server hardware to use for MySQL. I remember from the conference somebody (Pipes?) mentioning a particular Dell server with multiple disk RAID10 that could supposedly be had for about $6k but I completely misplaced the model number (Frank, did you get my email?).

I know that a multi-disk RAID array with a bunch of fast disks (15k RPM?) is probably the most important method of improving performance, followed by the amount of RAM, so I'm trying to find the best combination/balance of the two. However, server prices on the Internet range so

  [Read more...]