Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 69 Next 30 Older Entries

Displaying posts with tag: Hardware (reset)

How to improve InnoDB performance by 55% for write-bound loads
+2 Vote Up -0Vote Down

During April’s Percona Live MySQL Conference and Expo 2014, I attended a talk on MySQL 5.7 performance an scalability given by Dimitri Kravtchuk, the Oracle MySQL benchmark specialist. He mentioned at some point that the InnoDB double write buffer was a real performance killer. For the ones that don’t know what the innodb double write buffer is, it is a disk buffer were pages are written before being written to the actual data file. Upon restart, pages in the double write buffer are rewritten to their data files if complete. This is to avoid data file corruption with half written pages. I knew it has an impact on performance, on ZFS since it is transactional I always disable it, but I never realized how important the performance

  [Read more...]
GUUG Frühjahrsfachgespräch 2014: CfP ends on May 31st!
Employee +0 Vote Up -0Vote Down

The German Unix User Group (GUUG) will hold their annual conference "Frühjahrsfachgespräch" on September 23-26 this year (I know, not really "Frühjahr" anymore, but this is how it is).

The Call for Presentations is still open until May 31st. Talks can be proposed in German and English, and there are slots for longer tutorials as well.

The range of possible topics is broad, so if you think you have anything interesting to share with a very passionate and technical audience of sysadmins and developers, here are some suggestions:

  • Operating Systems/Applications: architectures, privilege concepts, new developments, administration, mobile systems
  • Relevant new OS Kernel
  [Read more...]
Linus on Instantiation and Armadaification
+0 Vote Up -1Vote Down

I feel a sense of pride when I think that I was involved in the development and maintenance of what was probably the first piece of software accepted into Debian which then had and still has direct up-stream support from Microsoft. The world is a better place for having Microsoft in it. The first operating system I ever ran on an 08086-based CPU was MS-DOS 2.x. I remember how thrilled I was when we got to see how my friend’s 80286 system ran BBS software that would cause a modem to dial a local system and display the application as if it were running on a local machine. Totally sweet.

When we were living at 6162 NE Middle in the nine-eight 292, we got an 80386 which ran Doom. Yeah, the original one, not the fancy new one with the double barrel shotgun, but it would probably run that one, too.

  [Read more...]
A Few Thoughts on OSCon and the Open Source Community
+1 Vote Up -0Vote Down

This past week I attended OSCon, the annual conference for open source’s true believers. And there was a religious fervor in the air, particularly from the point of view of someone more accustomed to Oracle conferences.

And if open source is the religion, proprietary closed-source companies are the devil. That having been said, I was surprised how virtually all large companies were demonized. Even long-time defenders of open source like IBM were ignored at best. That didn’t prevent them from coming though, with Microsoft and HP in particular with high-profile sponsorships and PR offensives that didn’t seem to have much influence with the crowd.

The companies generating buzz were the small companies built around development of their own open source products. There are a surprising number of them out

  [Read more...]
Hardware Components Failures – Survey Results
+1 Vote Up -0Vote Down

When preparing for the the IOUG Collaborate 12 deep dive on deploying Oracle Databases for high Availability, I wanted to provide some feedback on what hardware components are failing most frequently and which ones are less frequently. I believe I have reasonably good idea about that but I thought that providing some more objective data would be better. I couldn’t find and results of a more scientific research so I decided to organize a poll. This blog post shows the results and I promised to share it with several groups.

The results are also in the presentation material but it might be hidden deep into 100+ slides so here is the

  [Read more...]
Should RAID 5 be used in a MySQL server?
+1 Vote Up -0Vote Down

Usually the answer should be “no!”. RAID level 5 is hardly ever a good choice for any database storage. It comes with a very high overhead as each write turns into a sequence of four physical I/O operations, two reads and two writes, in order not only to update a data block, but also to re-calculate and update the corresponding checksum block. The resulting penalty is not just slower writes. The extra operations mean the storage I/O capacity is reduced too.

Another disadvantage of using RAID 5 could be its very poor performance when it works in degraded mode. In such configuration a disk failure means some data was actually lost, but RAID 5 can rebuild the missing pieces on-the-fly as requests arrive. But reconstructing blocks is nowhere near as efficient as just reading them from disk.

In most cases using alternative

  [Read more...]
Relearn about your battery
+0 Vote Up -0Vote Down

Some days ago I got a call from our support engineer on duty that MySQL on one of our database servers was lagging more than 1000 seconds behind in replication and the server got kicked out of the pool because of the delay. He was unable to find out why and there was absolutely nothing in the mysql log files. When I got the call it was still lagging behind but the lag was slowly decreasing again.

After a quick peek in all our monitoring systems I isolated it to this message:
Cache Battery 0 in controller 0 is Charging (Ready) [probably harmless]
Apparently not that harmless! :P

Obviously we did encountered this situation a couple of times before but apparently there was no detection on this machine.

The relearn cycle happens every 90 days and gets first scheduled when the machine gets powered on. Now imagine this happening in a

  [Read more...]
Got open source cloud storage? Red Hat buys Gluster
+0 Vote Up -0Vote Down

Red Hat’s $136m acquisition of open source storage vendor Gluster marks Red Hat’s biggest buy since JBoss and starts the fourth quarter with a very intersting deal. The acquisition is definitely good for Red Hat since it bolsters its Cloud Forms IaaS and OpenShift PaaS technology and strategy with storage, which is often the starting point for enterprise and service provider cloud computing deployments. The acquisition also gives Red Hat another weapon in its fight against VMware, Microsoft and others, including OpenStack, of which Gluster is a member (more on that further down). The deal is also good for Gluster given the sizeable price Red Hat is paying for the provider of open source, software-based, scale-out storage for unstructured data and also as validation of both open source and software in

  [Read more...]
CodeBits - An event of competitive innovation
+1 Vote Up -0Vote Down
It was my pleasure and privilege to attend Codebits in 2009. As Roland Bouman says, its talk choice method is based on public voting, and therefore everyone cha have contribute to the schedule.But that is not the main reason for attending this extraordinary event. It is not just a conference. It's an innovation fest. For 1 and 1/2 days, it's a conference, where the speakers are encouraged to bring to their audience the most innovative and inspiring talks. In the afternoon of the second day, the event becomes a competition, where the teams that have registered will have 24 hours to bring a project to completion, and they have  [Read more...]
Aligning IO on a hard disk RAID – the Benchmarks
+3 Vote Up -0Vote Down

In the first part of this article I have showed how I align IO, now I want to share results of the benchmark that I have been running to see how much benefit can we get from a proper IO alignment on a 4-disk RAID1+0 with 64k stripe element. I haven’t been running any benchmarks in a while so be careful with my results and forgiving to my mistakes

The environment

Here is the summary of the system I have been running this on (for brevity I have removed some irrelevant information):

# Aspersa System Summary Report ##############################
    Platform |
  [Read more...]
Aligning IO on a hard disk RAID – the Theory
+4 Vote Up -0Vote Down

Now that flash storage is becoming more popular, IO alignment question keeps popping up more often than it used to when all we had were rotating hard disk drives. I think the reason is very simple – when systems only had one bearing hard disk drive (HDD) as in RAID1 or one disk drive at all, you couldn’t really have misaligned IO because HDDs operate in 512-byte sectors and that’s also the smallest amount of disk IO that systems can do. NAND flash on the other hand can have a page size of 512-bytes, 2kbytes or 4kbytes (and often you don’t know what size it is really) so the IO alignment question becomes more relevant.

It was and still is, however, relevant with HDD RAID storage – technology we have been using for many years – when there’s striping like in RAID0, 5, 6 or any variation of them (5+0, 1+0, 1+0+0

  [Read more...]
So you want to run MySQL on SSDs?
+0 Vote Up -0Vote Down

Here’s why I do: it’s time for me to build a new master database server. Our current main slave is too underpowered to be handle our entire load in an emergency, which means that our failover situation isn’t that great. I’ll replace the master with something new and shiny, make some performance improvements while I’m at it, and the old master will work just fine in an emergency.

For IO intensive servers, I conserve space and electricity by using 1U machines with 6 or 8 2.5″ drives.

I’d normally buy 8 Seagate Savvio 15K SAS drives and set them up as a RAID 10 array. This would run me about $1850.

We’re pretty frugal when it comes to our technology budget and I can’t really stomach spending that kind of money to effectively get 550 GB of redundant, fast magnetic disk storage. SATA

  [Read more...]
dbbenchmark.com – now supporting MySQL on OSX 10.6
+2 Vote Up -0Vote Down

Just a quick note to let everyone know that our new benchmarking script now supports OSX 10.6 on Intel hardware. That means you can run one simple command and get all of the sequential and random INSERT and SELECT performance statistics about your database performance. As usual the script is open source and released under the new BSD license. Give is a try by downloading now! See the download page for more details.

dbbenchmark.com – Benchmarking script now available
+0 Vote Up -0Vote Down

You can download the first release of the benchmarking script here: http://code.google.com/p/dbbenchmark/

Please read the README file or consult the Support page before running the benchmarks.

Four short links: 1 July 2010
+0 Vote Up -0Vote Down

  • Conflict Minerals and Blood Tech (Joey Devilla) -- electronic components have a human and environmental cost. I remember Saul Griffith asking me, "do you want to kill gorillas or dolphins?" for one component. Now we can add child militias and horrific rape to the list. (via Simon Willison)
  • Meteor -- an open source HTTP server that serves streaming data feeds (for apps that need Comet-style persistent connections). (via gianouts on Delicious)
  • Hobby King RC Store -- online
  •   [Read more...]
    mount: /dev/sdb1 already mounted or /mysql busy
    +0 Vote Up -0Vote Down

    We added a 500GB 7.2K SATA/300 Hitachi Deskstar E7K500 16MB disk to one of our dev servers and partitioned using fdisk and formatted the partition with etx3. When we tried mounting the same, we got the following error :

    [root@xyz user]# mount -t ext3  /dev/sdb1 /mysql
    mount: /dev/sdb1 already mounted or /mysql busy

    lsof didn’t provide any open files that might be linked to this problem or there was any “famd” running. Finally doing the following steps to remove the logical devices from the device-mapper driver helped us fix the problem.

    [root@xyz user]# dmsetup ls
    ddf1_44656c6c202020201028001510281f033832b7a2f6678dab   (253, 0)
    ddf1_44656c6c202020201028001510281f033832b7a2f6678dab1  (253, 1)

    [root@xyz user]# dmsetup remove

      [Read more...]
    change accelerator cache ratio
    +0 Vote Up -0Vote Down

    I was given the task of checking the array accelerator cache ratio and see if it was set to optimal levels. Our ideal preference was a read/write ratio of 0/100.

    The machine configuration is HP DL180 G5, 2 x Xeon L5420 2.50GHz, 15.7GB / 16GB 667MHz DDR2, 6 x 300GB-15K SAS.This machine was running mysql 5.1.36 using the innodb plugin.

    The command line utility to check the controller configuration is “hpacucli”. Navigating using hpacucli is very straight forward.

    “ctrl all show config detail” Will give you the entire controller configuration.

    => ctrl all show config detail

    Smart Array P400 in Slot 5
    Bus Interface: PCI
    Slot: 5
    Serial Number: P61630K9SW31NL
    Cache Serial Number: PA82C0J9SW02H1
    RAID 6 (ADG) Status: Enabled
    Controller Status: OK

      [Read more...]
    RAID Controllers Cache Management – Missing Features
    +4 Vote Up -0Vote Down
    We all know how important hardware RAID controllers are in today’s data storage performance especially when dealing with large data sets. If we look at the trend from now to couple of years back; they really evolved rapidly with lot of useful features and their usage also grown as most of the new servers [...]
    Data Store, Software and Hardware – What is best
    +3 Vote Up -0Vote Down
    Other day we had a small discussion about data stores and hardware; and which one drives the other when it comes to data storage solution, rather it is a hard discussion as both on its own are bigger entities; and one can not easily conclude as it depends on use cases and actually speaking data [...]
    Dell MD1120 Storage Array Performance
    +2 Vote Up -0Vote Down
    Here is some file IO performance numbers from DELL MD1120 SAS storage array. Last year I did the same test with HP P800 storage array and numbers were impressive. But when it comes to this high end storage array, few surprises.  Before getting into actual details; lets see the test stats and configuration details. System Configuration: DELL [...]
    Ravelry Runs On – 2010
    +0 Vote Up -0Vote Down

    I guess that it’s time for the 3rd annual “Ravelry Runs On” roundup. The last two were in March 2008 and March 2009.

    This year, our traffic increased by 50% to 5,000,000 page views and 15 million Rails requests per day. We made very few changes to our architecture in 2009 but we did add a new master database server after our working set of data outgrew our memory and IO capacity.

    This summary is more detailed then the last two and I’ve broken it up into rough sections.

    Physical Network

    We own our own servers and colocate then in a datacenter here in Boston. The datacenter provides us with a cooled

      [Read more...]
    Four short links: 1 December 2009
    +0 Vote Up -0Vote Down

  • Apertus -- open source cinema camera. (via joshua on Delicious)
  • A Survey of Collaborative Filtering Techniques -- From basic techniques to the state-of-the-art, we attempt to present a comprehensive survey for CF techniques, which can be served as a roadmap for research and practice in this area. (via bos on Delicious)
  • Drizzle Replication using RabbitMQ as Transport -- we're watching the growing use of message queues in web software, and here's an interesting application. (via sogrady on
  •   [Read more...]
    Four short links: 4 November 2009
    +0 Vote Up -0Vote Down

  • ChipHacker -- collaborative FAQ site for electronics hacking. Based on the same StackExchange software as RedMonk's FOSS FAQ for open source software.
  • Democracy Live -- BBC launch searchable coverage of parliamentary discussion, using speech-to-text. One aspect we're particularly proud of is that we've managed to deliver good results for speech-to-text in Welsh, which, we're told, is unique. I think of this as the start of a They Work For You for video coverage. I'd love to be able to scale this to local government coverage, which is disappearing as local newspapers turn into delivery mechanisms
  •   [Read more...]
    MySQL and hardware information
    +8 Vote Up -6Vote Down

    People often ask “what’s the best hardware to run a database on?” And the answer, of course, is “it depends”. With MySQL, though, you can get good performance out of almost any hardware.

    If you need *great* performance, and you have active databases with a large data set, here are some statistics on real life databases — feel free to add your own.

    We define “large data set” as over 100 Gb, mostly because smaller data sets have an easier time with the available memory on a machine (even if it’s only 8 Gb) and backups are less intrusive — InnoDB Hot Backup and Xtrabackup are not really “hot” backups, they are “warm” backups, because there is load on the machine to copy the data files, and on large, active servers we have found that this load impacts query performance. As for how active

      [Read more...]
    Advanced Squid Caching in Scribd: Hardware + Software Used
    +0 Vote Up -0Vote Down

    After the previous post in this caching related series I’ve received many questions on hardware and software configuration of our servers so in this post I’ll describe our server’s configs and the motivation behind those configs.

    Hardware Configuration

    Since in our setup Squid server uses one-process model (with an asynchronous requests processing) there was no point in ordering multi-core CPUs for our boxes and since we have a lots of pages on the site and the cache is pretty huge all the servers ended up being highly I/O bound. Considering these facts we’ve decided to use the following hardware specs for the servers:

    CPU: One pretty cheap dual-core Intel Xeon 5148 (no need in multiple cores or really high frequencies –

      [Read more...]
    Sun/Intel X-25e 4 Disk Raid 10 tests - part 2
    +0 Vote Up -0Vote Down

    So lets test some different configurations and try and build some best practices around Multiple SSD’s:

    Which is better? Raid 5 or Raid 10?

    As with regular disks, Raid 10 seems to performance better ( accept for pure reads ).  I did get a lot of movement test to test like with the 67% read test -vs- the 75% or 80% tests. But all in all RAID 10 seemed to be the optimal config.

    Should you enable the controller cache? One of the things I have found in my single drive tests is that “dumb” controllers tend to give better performance numbers then “smart” controllers. Really expensive controllers tend to have extra logic to compensate for the limitations of traditional disk. So I

      [Read more...]
    x-25e, 25% reduction in random writes…
    +1 Vote Up -0Vote Down

    So in my previous post I showed some benchmarks showing a large drop off in performance when you fill the x-25e. I wanted to followup and say this: even if you do everything correctly ( i.e. leave 50%+ space free, disable controller cache etc ) you may still see a drop in performance if your workload is heavily write skewed.  To show this I ran a 100% random read sysbench fileio test over a 12GB dataset (37.5% full ) , the tests were run back-to-back over a several hours , here is what we see:

    *Note the scale is a little skewed here ( i start at 2500 reqs ).

    Each data point represents 2 million IO’s, so somewhere after about 6 million IO’s we start to drop.  At the end it looks like we stabilize around2900-3000 requests per second, an overall drop of about 25%.

    Intel X-25e and Mysql Part 1b: Don’t let your Drive Over Eat!
    +4 Vote Up -0Vote Down

    The plan was only to do two quick posts on RAID Performance on the X-25e, but this was compelling enough to post on it’s own.  So in part I Mark Callaghan asked hey what gives with  the SLC Intel’s single drive random write performance,  It’s  lower then the MLC drive.   To be completely honest with you I had overlooked it, after all I was focusing on RAID performance.  This was  my mistake because this is actually caused by one of the Achilles heals of most flash on the market today, crappy performance when you fill more of the drive.  I don’t really know what the official title for it is but I will call it “Drive Overeating”.

    Let me try and put this simply:  a quick trick most vendors use to push better random write #’s and help wear leveling is to not fully erase

      [Read more...]
    Sun/Intel X-25e 4 Disk Raid 10 tests - part 1
    +2 Vote Up -0Vote Down

    Everyone loves SSD.  It’s a hot topic all around the MySQL community with vendors lining up all kinds of new solutions to attack the “disk io”  problem that has plagued us all for years and years.  At this year’s user conference I talked about SSD’s and MySQL.   Those who follow my blog know I love IO and I love to benchmark anything that can help overcome IO issues.  One of the most exciting things out their at this point are the Intel x-25e drives.  These bad boys are not only fast but relatively inexpensive.  How fast are they?  Let’s just do a quick bit of review here and peak at the single drive #’s from sysbench.    Here you can see that a single X25-e outperforms all my other single drive test.

      [Read more...]
    My Really High DBT2 Scores
    +1 Vote Up -0Vote Down

    Pre-UC I put out a teaser on some dbt2 scores in the 50K range.   I mentioned and showed the graphs during my SSD session, but I thought I would show them here for those who skipped the UC or did not attend my session.  Basically what most people consider to be a classic “CPU Bound” workload where all of your data easily fits into memory can also see benefits from moving to SSD’s. Remember just because everything fits into memory doesn’t mean your not going to be doing some operations to disk ( logging, flushes, etc ). Take a look:

    Test TPM % Improvement Regular Disk BBU (5.1.33) 46106.44 NA SSD WO/Drive Cache (5.1.33) 50606.82 9.76% SSD W/Drive Cache (5.1.33) 55793.47 21.01% SSD WO/Drive Cache (5.4) 66286.95 43.77% SSD W/Drive Cache (5.4) 70264.66

      [Read more...]
    Showing entries 1 to 30 of 69 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.