Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 42 Next 12 Older Entries

Displaying posts with tag: matt (reset)

Speaking @ Percona Live in London Next Week!
+0 Vote Up -0Vote Down

A quick note, I am speaking at Percona live in London next week… its should be a rip roaring time. I have two topics I am speaking on.

The first is on building a MySQL Data Access Layer with Ruby and Sinatra. While this may seem a bit odd, its actually very cool and useful. With

The second presentation, How I learned to stop worrying and Love Big Data should be fun. I promise the following: An interesting take on Big data, Dr Strangelove, Simpson’s Humor, Daleks, wacky signs, Darth Vader, and a Monty Python reference… oh yeah this should be fun.

MySQL Bootcamp at Collaborate 2011 pt 2
+3 Vote Up -0Vote Down

Hi All,

I am going through some of the sessions for IOUG’s Collaborate 2011 Conference and trying to fill in slots for the bootcamp, and while we have some great sessions we could use a few more sessions. Specifically I would love to get a couple of sessions on a few the following topics:

  • InnoDB in General
  • InnoDB Internals & scalability
  • General Overview of available Storage Engines (Pros/Cons)
  • MySQL options for Very Large databases ( ala Partitioning, Sharding )
  • MySQL Monitoring Options
  • NDB Cluster

These are just a few suggestions, please feel free to submit any topic thats related to MySQL…  even if its not a fit for the bootcamp, there is a MySQL track that it would fit into.  IOUG extended the MySQL Deadline until next Monday for us, so let’s get some more papers in!  You can submit

  [Read more...]
Helping to Organize a MySQL Bootcamp @ IOUG’s COLLABORATE 11
+2 Vote Up -0Vote Down

I am helping IOUG Organize a MySQL bootcamp at their Collaborate conference in Orlando. This is actually a great opportunity to reach out to a lot of Oracle talent looking for more information and training on MySQL. An IOUG Bootcamp is compiled from several 1 hour technical sessions starting from introductory level topics on day 1 to advanced topics the final day. The idea is this format will help those not familiar with MySQL, get a crash course in MySQL while also providing people with a wide range of targeted sessions that they can come in an out of as they see fit. It’s like the tag line of most Carnivals, Fun for all ages…

The reason I am posting is we are going to need lots of help from you. We are looking for speakers for the conference at the moment… you can submit your papers here:

  [Read more...]
Great to see everyone at the UC
+0 Vote Up -0Vote Down

It was awesome to see everyone at the 2010 mysql UC. Sorry if I did not get a chance to chat with everyone, time just flew by! I had great turn out for my two sessions and had a lot of great conversations with people. If people are looking for my slides they are posted on the User Conference Website here: http://en.oreilly.com/mysql2010/public/schedule/speaker/75377 .. Thanks Everyone!

More Debate, More Flame, More Choosing the correct tool for the job
+1 Vote Up -0Vote Down

You have to love all the debating going on over NOSQL -vs- SQL don’t you? With my UC session on choosing the right data storage tools ( does this sound better then SQL-vs-NoSQL?) I have been trying to stay current with the mood of the community so i can make my talk more relevant. Today I was catching up on reading a few blogs posts and I thought I would pass along these two: Pro SQL and Pro NoSQL … these represent the two very different views on this subject. (Note I think there are misleading facts and figures in these that should be flushed out more, but they are a good sample of what I am talking about).

  [Read more...]
New Benchmark I am working on that tests MYSQL -vs- NOSQL
+2 Vote Up -0Vote Down

I am giving a talk in a couple of weeks at the 2010 MySQL User Conference that will touch on use cases for NOSQL tools -vs- More relational tools, the talk is entitled “Choosing the Right Tools for the Job, SQL or NOSQL”. While this talk is NOT supposed to be a deep dive into the good, bad, and ugly of these solutions, rather a way to discuss potential use cases for various solutions and where they may make a lot of sense, being me I still felt a need to at least do some minor benchmarking of these solutions. The series of posts I wrote last year over on mysqlperformanceblog.com comparing Tokyo Tyrant to both MySQL and Memcached was fairly popular. In fact the initial set of benchmark scripts I used for that series actually has been put to good use since then

  [Read more...]
Huge Data -vs- The Database, how the industry is adapting in the face of the data explosion
+0 Vote Up -0Vote Down

Note This Article was actually written back in May after the UC at the request of Linux Magazine, through a series of events It went unpublished. Between then and now Jeremy ended up doing a great job covering most of the topics, so in the end it was unneeded. Now I had this completed article and thought, what should I do with it? In the I decided to publish them here. Also note I did update a few items.

As more companies move to MySQL and the demands for data increase, we push the bounds of the database further. The challenges large Web properties (who have pioneered many of the large MySQL deployments) faced when they stored 50GB of data and had 5,000 users were nothing like the challenges of storing 500GB of data supporting 100,000 users. Today, as we are seeing more and more 10+TB-sized datasets

  [Read more...]
You know you have really screwed up when …
+3 Vote Up -0Vote Down

You crash MySQL/Innodb and you trace the error to a function that says:

“This function checks the consistency of an index page when we do not
know the index. This is also resilient so that this should never crash
even if the page is total garbage. ”

Oops… I guess its not that resilient after all:)

Testing Waffle
+0 Vote Up -0Vote Down

So Ronald Bradford has been getting some EC2 AMI’s setup for Waffle the last couple of days. Really he is the first person to work with us on testing out the 0.5 cream release. It’s funny but when you are heads down in the code you tend to overlook some of the easy and obvious bugs. Just by having him walk through the install we have opened up several new bugs and come up with some new Idea’s on how to make things easier in the future. I hope by early next week we can have some of annoying bugs fixed and code pushed to launchpad. When that happens I would encourage others to give it a whirl and help us make Waffle Grid better performing, more user friendly, and more stable.

You can check out some of the bugs here.

Thanks to Ronald for his help on this!

5 Minute DBA MyISAM Example Config Files
+0 Vote Up -0Vote Down

I have been asked this numerous times ever since I posted my InnoDB sample configuration files, Do I have sample configuration parameters for a MyISAM setup? We are seeing less and less people use MyISAM, but it is still popular ( especially in prepackaged form i.e. wordpress). So i figured why not adjust my sample InnoDB configs and make them suitable for MyISAM.

The biggest limitation to MySIAM is used to be the default key buffer only can could only be sized up to 4GB ( This was fixed in 5.0.52) . While you can create separate key buffers and assign indexes to them, it’s not very common ( Common as in present in low-end shops who need 5 minute dba help) in part because I think people do not fully understand it and you do have to plan for it. Setting up secondary caches is a database/application specific setup, it can

  [Read more...]
Leaving Sun/MySQL but not the community
+0 Vote Up -0Vote Down

I thought I would put out a quick note about this right now, I am going to be leaving Sun next week. I do have some plans which will start to materialize over the next few weeks and months so stick around for that. Even though I am leaving the company, I will still be lurking and working on all think database and performance related. This should not have any bearing on BigDBAHead.com or Waffle Grid, hopefully this will only mean I will post more and spend more time doing crazy and cool things!

I wanted to thank everyone at MySQL who I worked with and learned from over the past few years.

WaffleGrid: Cream Benchmarks, stable and delivering a 3x boost
+0 Vote Up -0Vote Down

Lets get down to how the latest version of Waffle Grid performs.

Starting off simple lets look at the difference between the wafflegrid modes. As mentioned before the LRU mode is the “classic” Waffle Grid setup. A page is put into memcached when the page is removed from the buffer pool via the LRU process. When a page is retrieved from memcached it is expired so its no longer valid. In the New “Non-LRU” mode when a page is read from disk, the page is placed in memcached. When a dirty page is flushed to disk, this page is overwritten in memcached. So how do the different modes perform?

4GB Memcached, Read Ahead Enabled TPM % Increase No Waffle 3245.79 Baseline Waffle LRU 10731.34 330.62% Waffle NoLRU 10847.52 334.20%

You can see here that with 100% of

  [Read more...]
WaffleGrid: 0.5 Cream Release
+0 Vote Up -0Vote Down

I wanted to let everyone know that we are releasing Waffle Grid 0.5 code named Cream. This release fixes the nasty secondary index bug that plagued the butter release. I have been running tests on this code base for about a week straight with no errors. While I think this release is much more stable I would remind everyone this is still not a fully GA release. This release includes the ability to choose the mode of Waffle grid. By setting innodb_memcached_enable to 1, we will push pages to memcached when a disk read is done or when a page write is done, setting this to 2 will enable the classic LRU mode. If you decide to set this to 1 ( non-lru) I would recommend using the standard memcached, as with previous versions the LRU mode works better with our slightly altered memcached ( expire from memcached on get ). I will be posting benchmarks and more details within the next couple of days. Right now you can

  [Read more...]
Be Wary of Large Log File Sizes
+3 Vote Up -0Vote Down

As some people have mentioned here and here Increasing the innodb log file size can lead to nice increases in performance. This is a trick we often deploy with clients so their is not anything really new here. However their is a caveat, please be aware their is a potentially huge downside to having large log file sizes and that’s crash recovery time. You trade real-world performance for crash recovery time. When your expecting your shiny Heartbeat-DRBD setup to fail-over in under a minute this can be disastrous! In fact I have been some places were recovery time is in the hours. Just keep this in mind before you change your settings.

Waffle: Progress and a Rearchtecture?
+1 Vote Up -0Vote Down

So I spent several hours over the last few days on the Secondary index bug. Out of frustration I decided to try and bypass the LRU concept all together and try going to a true secondary page cache. In standard Waffle a page is written to memcached only when it is expunged ( or LRU’d ) from the main buffer pool. This means anything in the BP should not be in memcached. Obviously with this approach we missed something, as Heikii pointed out in a comment to a previous post, it seems likely we are getting an old version of a page. Logically this could happen if we do not correctly expire a page on get or we bypass a push/lru leaving an old page in memcached to be retrieved later on.

So I was thinking why not bypass the LRU process? While I feel this is the most efficient way to do this, its not the only way. I modified innodb to use the default LRU code and then modified the page get to push to

  [Read more...]
Waffle: The Mystery Continues
+0 Vote Up -0Vote Down

So I spent the weekend looking at places where we may have missed something in the code for waffle. You can actually see some of the stuff I tried in the bug on launchpad about this, but the weird thing is the very last thing I tried. As I took a step back and looked at the problem ( secondary index corruption ) and our assumption that we “missed” something, I decided to find the place where pages are written to disk and to push to memcached from here as well as from the LRU. With the double write buffer enabled that place should be buf_flush_buffered_writes. By pushing to memcached here we should eliminate the page that falls through the cracks of the LRU. Basically this should help ensure memcached has an exact copy of the data that exists on disk. The result? It failed with the same secondary index failure. This means:

a.)

  [Read more...]
Waffle: limiting the space ids being pushed to memcached
+2 Vote Up -0Vote Down

If you read Yves blog post about waffle yesterday we are seeing some weird gremlins in the system and could use some scoobey doo detective work if you have some ideas. The strange thing is it only exhibits under high load. So it really seems like we may have missed some background cleanup process that accesses or removes pages from disk or the buffer pool without going through the functions we call waffle in (buf_LRU_search_and_free_block & buf_read_page_low ).

One of the idea’s I had was trying to narrow the scope of what’s being pushed and read form Memcached. Even though I am using file per table, system tablespace pages are still making it in and out of memcached. I thought if we missed something maybe it was here ( even though I could not find it in the code ). I mean cleaning up undo or internal data would seem like a logical place to

  [Read more...]
Sun/Intel X-25e 4 Disk Raid 10 tests - part 2
+0 Vote Up -0Vote Down

So lets test some different configurations and try and build some best practices around Multiple SSD’s:

Which is better? Raid 5 or Raid 10?

As with regular disks, Raid 10 seems to performance better ( accept for pure reads ).  I did get a lot of movement test to test like with the 67% read test -vs- the 75% or 80% tests. But all in all RAID 10 seemed to be the optimal config.

Should you enable the controller cache? One of the things I have found in my single drive tests is that “dumb” controllers tend to give better performance numbers then “smart” controllers. Really expensive controllers tend to have extra logic to compensate for the limitations of traditional disk. So I

  [Read more...]
x-25e, 25% reduction in random writes…
+1 Vote Up -0Vote Down

So in my previous post I showed some benchmarks showing a large drop off in performance when you fill the x-25e. I wanted to followup and say this: even if you do everything correctly ( i.e. leave 50%+ space free, disable controller cache etc ) you may still see a drop in performance if your workload is heavily write skewed.  To show this I ran a 100% random read sysbench fileio test over a 12GB dataset (37.5% full ) , the tests were run back-to-back over a several hours , here is what we see:

*Note the scale is a little skewed here ( i start at 2500 reqs ).

Each data point represents 2 million IO’s, so somewhere after about 6 million IO’s we start to drop.  At the end it looks like we stabilize around2900-3000 requests per second, an overall drop of about 25%.

Intel X-25e and Mysql Part 1b: Don’t let your Drive Over Eat!
+4 Vote Up -0Vote Down

The plan was only to do two quick posts on RAID Performance on the X-25e, but this was compelling enough to post on it’s own.  So in part I Mark Callaghan asked hey what gives with  the SLC Intel’s single drive random write performance,  It’s  lower then the MLC drive.   To be completely honest with you I had overlooked it, after all I was focusing on RAID performance.  This was  my mistake because this is actually caused by one of the Achilles heals of most flash on the market today, crappy performance when you fill more of the drive.  I don’t really know what the official title for it is but I will call it “Drive Overeating”.

Let me try and put this simply:  a quick trick most vendors use to push better random write #’s and help wear leveling is to not fully erase

  [Read more...]
Sun/Intel X-25e 4 Disk Raid 10 tests - part 1
+2 Vote Up -0Vote Down

Everyone loves SSD.  It’s a hot topic all around the MySQL community with vendors lining up all kinds of new solutions to attack the “disk io”  problem that has plagued us all for years and years.  At this year’s user conference I talked about SSD’s and MySQL.   Those who follow my blog know I love IO and I love to benchmark anything that can help overcome IO issues.  One of the most exciting things out their at this point are the Intel x-25e drives.  These bad boys are not only fast but relatively inexpensive.  How fast are they?  Let’s just do a quick bit of review here and peak at the single drive #’s from sysbench.    Here you can see that a single X25-e outperforms all my other single drive test.

  [Read more...]
Boy am I a slacker!
+0 Vote Up -1Vote Down

Must of really been worn out post UC because my experiments and blogging have dipped to all time lows… but rest assured I am on my way back. I will be following up posting some posts on my Sun/Intel SSD tests I did for my presentation at the UC. This will give those who did not go to the UC a chance to hear me babble on about SSD’s some more ( OH NO, not more babble about SSD’s! ).

Plus of course their is Waffle. Right now I am perplexed as ever by our mysterious index issue… and its been bothering me for weeks! I think I will post some details and look for external opinions very soon.

My Really High DBT2 Scores
+1 Vote Up -0Vote Down

Pre-UC I put out a teaser on some dbt2 scores in the 50K range.   I mentioned and showed the graphs during my SSD session, but I thought I would show them here for those who skipped the UC or did not attend my session.  Basically what most people consider to be a classic “CPU Bound” workload where all of your data easily fits into memory can also see benefits from moving to SSD’s. Remember just because everything fits into memory doesn’t mean your not going to be doing some operations to disk ( logging, flushes, etc ). Take a look:


Test TPM % Improvement Regular Disk BBU (5.1.33) 46106.44 NA SSD WO/Drive Cache (5.1.33) 50606.82 9.76% SSD W/Drive Cache (5.1.33) 55793.47 21.01% SSD WO/Drive Cache (5.4) 66286.95 43.77% SSD W/Drive Cache (5.4) 70264.66

  [Read more...]
Performance Experts Panel Q&A
+1 Vote Up -0Vote Down

Had a fun time being on Percona’s performance panel late last night. For those who were there,  I was probably the only guy you did not recognize:)   The  panel of Ronald Bradford,  Mark Callaghan, Cary Millsap, Jay Pipes, Michael “Monty” Widenius, and myself spent the evening answering questions from the audience and of course drinking Monty’s famous black vodka.  While a lot of the questions drifted away from performance I think everyone had a good time and found the session to be informative.  Now the question i have is : does anyone have any good photo’s of the event?

Random notes from conversations at the UC
+2 Vote Up -0Vote Down

People always have conversations at the UC that spark ideas for new projects or features, this is my random collection of features or items I need to look into.  Most people can ignore it… but if your interested in what goes on in my head here ya go:

General Innodb:

  • I started looking into this before, but I was reminded of it again today…  it would be useful to setup separate innodb buffer pools like the key caches in MyISAM.
  • maybe an easy hack to put in a tiered LRU based on file per table.  What I mean by this is internally the internal BP in inno actually uses the space id ( file per table will have a new ID per table )…  this should make it feasible to ( maybe easy ) to move certain spaceid’s up or down the internal LRU list  This maybe an interesting feature, not exactly like pinning a table into the BP but close.    Basically, if a table is accessed
  [Read more...]
TIME TO PANIC!
+0 Vote Up -0Vote Down

We have 3 presentations today and you may not have been thinking of attending!  What’s wrong with you.

Oh… did you think I was going to say something else?

Stop by say Hi, seek us out after our sessions and introduce yourself.  I would love for you to stop by  give us your idea’s on Waffle, Bigdbahead, MySQL, benchmarks, Life, Oracle, Baseball, or anything else on your mind.

Don’t Panic…
+1 Vote Up -0Vote Down

Remember the what’s printed on the front of the Guide… “Don’t Panic”.

First we all get hit hard with News like this…  So we go through lots of stages. Panic, dear, acceptance, etc.  They all come and everyone is left asking now what.

First… look around at and get reassurance from others…

Sheeri has a decent post on her take on the news here.

Second:

As I mentioned earlier, it’s way to early to speculate on what’s going to happen.  Integration can not begin until all the regulatory stuff get handle, which could take months and months.  So their is going to be a really long quiet period of time.  For us internal folks this will be a scarey time of course.  So anything you hear now

  [Read more...]
Interesting Feel this morning @ the UC
+1 Vote Up -0Vote Down

Everyone who was up @ 7:30am seems to feel tired and subdued.  Everyone I heard from is worried about what this means for MySQL.  Some people have taken to extreme joking running around with cracks about the evil empire and “hail Larry”.    It’s a very tense mood all in all.   The feeling seems to be that the announcement was timed to have maximum effect, others seem to think it was timed so they can all bring us over to the assimilation machine and get our implants embedded in our brains.  I am sure you are going to be hearing a lot of fear & uncertaintity, rumors and the like over the week but their is still a long way to go, and nothing will be worked out officially for months.

That being said it does feel like the ” The end of the world as we know it”  … welcome to black monday.

Counting down the days before UC Time!!
+0 Vote Up -0Vote Down

One last reminder, we ( Yves and I ) will be appearing all over the place at the UC. Here is a quick run down of our schedule:

  • Tuesday 10:50 am - 11:35 pm (Free MySQL Camp) I will be presenting “Learning from others’ MySQL Performance Mistakes” … a journey into the sometimes silly things we all end up doing.
  • Tuesday 2-3pm Yves and I are co-presenting “Distributed Innodb Caching with memcached” … this will be an intro into the Waffle Grid project, what it’s about and what it means to you.
  • Tuesday 4:25 pm - 5:10 pm (Free MySQL Camp) Yves will be presenting “NBD (MySQL Cluster) performance tuning and pitfalls”
  • Wednesday 2-3pm I will be presenting: “SAN Performance on a Internal Disk Budget: The Coming Solid State Disk Revolution” … the title sounded better when I
  [Read more...]
Tease me, SUN SSD Benchmarks
+0 Vote Up -0Vote Down

Only a little over a week before the User conference and I am still burning the midnight oil to get as much information for my presentations as possible. I thought I would tease you a bit here. What do you get when you put 4 Intel X-25E’s ( Sun branded) SSD’s running RAID10 in a Sun 4450 and run the sysbench fileio test on it?

NO CTL, NO DRIVE
Hardware NO CTL, W DRIVE
Hardware W CTL, NO DRIVE
Hardware W CTL, W DRIVE
Hardware NO CTL, NO DRIVE
Software 50% Reads 3449.25 7744.36 2585.44 8656.63 3714.53 67% Reads 4460.67 8696.23 4169.18 9325.29 4646.03 75% Reads 5538.94 10016.72 5233.61 9942.23 5930.73 80% Reads 6886.81 12385.5 6194.27 10260.07 7378.55 83% Reads 7067.62 12895.92 6958.61 10247.36 7911.37 100% Reads 16608.69 16438.89 10814.62 11064.44 18050.14 100% Writes 1983.76 6469.18 1904.77 4328.93 1939.74

*NO CTL or W CTL : No






  [Read more...]
Showing entries 1 to 30 of 42 Next 12 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.