Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Previous 30 Newer Entries Showing entries 31 to 60 of 61 Next 1 Older Entries

Displaying posts with tag: ec2 (reset)

EC2/EBS single and RAID volumes IO benchmark
+2 Vote Up -0Vote Down

During preparation of Percona-XtraDB template to run in RightScale environment, I noticed that IO performance on EBS volume in EC2 cloud is not quite perfect. So I have spent some time benchmarking volumes. Interesting part with EBS volumes is that you see it as device in your OS, so you can easily make software RAID from several volumes.

So I created 4 volumes ( I used m.large instance), and made:

RAID0 on 2 volumes as:
mdadm -C /dev/md0 --chunk=256 -n 2 -l 0 /dev/sdj /dev/sdk

RAID0 on 4 volumes as:
mdadm -C /dev/md0 --chunk=256 -n 4 -l 0 /dev/sdj /dev/sdk /dev/sdl /dev/sdm

RAID5 on 3 volumes as:
mdadm -C /dev/md0 --chunk=256 -n 3 -l 5 /dev/sdj /dev/sdk /dev/sdl

RAID10 on 4 volumes in two steps:

mdadm






  [Read more...]
Three key things to know about moving MySQL into the cloud.
+4 Vote Up -0Vote Down

The question "what problems will I have when migrating to the cloud" gets asked often enough. If by cloud you mean Amazon EC2, then from a technical perspective there isn't much that changes. The biggest thing that changes is just how you pay your bill.

Having said that, there's still a few potential gotchas:

  • There are no Virtual IP addresses. Most High Availability tools (like MMM or DRBD+Heartbeat)
    work on the principal of having a floating IP address which is used for the application to connect to the current master. With EC2, you can't do this.
  • There's no customization of the memory. The maximum amount of memory you can have is 15GB, so some users with larger working sets may find this a limitation. If you look at the Dell online store, it costs $2094 to upgrade an R900

  •   [Read more...]
    Your opinion on EC2 and other cloud/hosting options
    +0 Vote Up -0Vote Down

    EC2 is nifty, but it doesn’t appear suitable for all needs, and that’s what this post is about.

    For instance, a machine can just “disappear”. You can set things up to automatically start a new instance to replace it, but if you just committed a transaction it’s likely to be lost: MySQL replication is asynchronous, EBS which is slower if you commit your transactions on it, or EBS snapshots which are only periodic (you’d have to add foo on the application end). This adds complexity, and thus the question arises whether EC2 is the best solution for systems where this is a concern.

    When pondering this, there are two important factors to consider: a database server needs cores, RAM and reasonably low-latency disk access, and application servers should be near their database server. This means you shouldn’t split app and db servers to

      [Read more...]
    451 CAOS Links 2009.06.12
    +0 Vote Up -0Vote Down

    Yahoo opens up Hadoop distribution. Microsoft and Novell claim customer wins. And more.

    Follow 451 CAOS Links live @caostheory

    The elephant in the room
    Plenty of news emerged form the Hadoop Summit this week, including Cloudera announced support for Amazon Elastic Block Storage (EBS) and introduced Sqoop, open source tool for importing databases into Hadoop, while Yahoo! Released! The! Yahoo! Distribution! Of! Hadoop! opening up its Hadoop developments to the wider community. As Savio Rodrigues noted, there has been a surge in the number of contributors for the Hadoop project in the last year.

    Best of the rest
    # Novell and Microsoft


      [Read more...]
    SysBench on EC2: Size Matters
    +0 Vote Up -0Vote Down

    It been sometime since we benchmarked MySQL/Galera with sysbench, using it mostly for testing. Our recent visit to Percona Performance Conference showed that sysbench is probably most widely used tool for MySQL benchmarking in the community and besides it is the only benchmark I know that correctly measures response times. So I just gave it a shot with our 0.6 release.
    I ran OLTP test on 1-4 large EC2 instances. At first I tried 100K row table and it was good except that the deadlock rate was too high to my taste:

    nodes	users	trx/s	deadlks	95%lat
    --------------------------------------
    4	40	840	28.13	0.099
    4	60	866	86.34	0.150
    4	80	781	194.8	0.240
    

    Note how deadlock rate escalates with the number of concurrent connections. But what is 100K rows by modern standards? Kids play. So I tried 1M rows. And it just shows that Galera cluster is cut for big tables:


      [Read more...]
    Announcing Drizzle on EC2
    +1 Vote Up -0Vote Down

    I have published the very first sharable Drizzle Amazon Machine Image (AMI) for AWS EC2, based on the good feedback from my discussion at the Drizzle Developer Day on what options we should try.

    This first version is a 32bit Developer instance, showcasing Drizzle and all necessary developer tools to build Drizzle from source.

    What you will find on drizzle-ami/intrepid-dev32 - ami-b858bfd1

    Ubuntu 8.10 Intrepid 32 bit base server installation:

    • build tools
    • drizzle dependencies
    • bzr 1.31.1

    From the respective source trees the following software is available:

    • drizzle 2009.04.997
    • libdrizzle 0.0.2
    • gearman 0.0.4
    • memcached 1.2.8
    • libmemcached 0.28

    Drizzle has been configured

      [Read more...]
    Sun talks out Cloud: Open Cloud Platform
    Employee +0 Vote Up -0Vote Down

    Sun's Open Cloud Vision unveilled: Open Cloud Platform, an open infrastructure powered by Java, MySQL, OpenSolaris, and Open Storage software technologies.  Open APIs, Open formats and Open source.

    On March 18th, at CommunityONE aka CloudONE, Sun unveiled the open cloud platform for powering public and private clouds. We also  announced that we are building our own Public Cloud. This will include a Storage and Compute Cloud. Our Cloud will be compatible with Amazon S3 and EC2 at the API level. Meaning, we will provide S3 and EC2 compatibility APIs in addition to our own, hence enabling an easy migration from Amazon services to Sun Cloud. All clouds - public, private or hybrid, built on Sun's Open Cloud

      [Read more...]
    Sun talks out Cloud: Open Cloud Platform
    Employee +0 Vote Up -0Vote Down

    Sun's Open Cloud Vision unveilled: Open Cloud Platform, an open infrastructure powered by Java, MySQL, OpenSolaris, and Open Storage software technologies.  Open APIs, Open formats and Open source.

    On March 18th, at CommunityONE aka CloudONE, Sun unveiled the open cloud platform for powering public and private clouds. We also  announced that we are building our own Public Cloud. This will include a Storage and Compute Cloud. Our Cloud will be compatible with Amazon S3 and EC2 at the API level. Meaning, we will provide S3 and EC2 compatibility APIs in addition to our own, hence enabling an easy migration from Amazon services to Sun Cloud. All clouds - public, private or hybrid, built on Sun's Open

      [Read more...]
    Sun talks out Cloud: Open Cloud Platform
    Employee +0 Vote Up -0Vote Down

    Sun's Open Cloud Vision unveilled: Open Cloud Platform, an open infrastructure powered by Java, MySQL, OpenSolaris, and Open Storage software technologies.  Open APIs, Open formats and Open source.

    On March 18th, at CommunityONE aka CloudONE, Sun unveiled the open cloud platform for powering public and private clouds. We also  announced that we are building our own Public Cloud. This will include a Storage and Compute Cloud. Our Cloud will be compatible with Amazon S3 and EC2 at the API level. Meaning, we will provide S3 and EC2 compatibility APIs in addition to our own, hence enabling an easy migration from Amazon services to Sun Cloud. All clouds - public, private or hybrid, built on Sun's

      [Read more...]
    Cloud Computing - Executive Seminar
    +0 Vote Up -0Vote Down
    Tomorrow, I'll be attending the Executive Seminar on Cloud Computing at NASDAQ MarketSite (NY). Speakers include Dr. Werner Vogels and Mårten Mickos (ex-CEO of MySQL). Big thanks to Amazon and RightScale who were able to accommmodate my RSVP even when the registration had formally closed.

    I hope to be able to catch up with Mårten Mickos during the event. In case I do succeed in catching up, is there any question you want me to ask him? You can email me or post a comment.

    It's funny that the event site still shows Mårten's title as "SVP of Sun Microsystems’ Database Group."
    FathomDB: Database as a service, in the cloud
    +0 Vote Up -0Vote Down

    A lot of people are into the whole cloud computing scenario these days. However, no one has talked about offering DBA-like services in the cloud, all automated, so that startups don’t have to get their own DBAs.

    Enter FathomDB. They are poised to offer databases as a service (maybe they’ll charge per database - so you can in theory run both WordPress and Mediawiki, if you prefix wp_ and mw_ in your table creation, for example). They are using MySQL. They’ve also taken the worry of running a database out - they will backup, they will setup (so you don’t have to issue GRANT commands :P), and they will also monitor your databases for you.

    But what really takes the cake? The fact that they will also offer performance advisors. This totally reminds me of the MySQL

      [Read more...]
    OurDelta MySQL on EC2 - install
    +0 Vote Up -0Vote Down
    Summary:Arjen would give me an earful if I got this wrong or poorly worded."OurDelta produces enhanced builds for MySQL, with OurDelta and third-party patches, for common production platforms" from http://ourdelta.org/aboutOver the next series of articles I am going to put the many additions to the MySQL 5.0 baseline through their paces on Amazon EC2.Using a base CentOS 4.4 I had lying around on
    On Why Auto-Scaling in the Cloud Rocks
    +0 Vote Up -0Vote Down

    In high school, I had a great programmable calculator. I’d program it to solve complicated math and science problems “automatically” for me. Most of my teachers got upset if they found out, but I’ll always remember one especially enlightened teacher who didn’t. He said something to the effect of “Hey, if you managed to write software to solve the equation, you must thoroughly understand the problem. Way to go!”.

    George Reese wrote up a blog post over at O’Reilly the other day called On Why I Don’t Like Auto-Scaling in the Cloud. His main argument seems to be that auto-scaling is bad and reflects poor capacity planning. In the comments, he specifically calls

      [Read more...]
    Test your Solaris Application on MySQL
    Employee +0 Vote Up -0Vote Down

    There are a couple of easy ways to test your Solaris application with MySQL ("the world's most popular opensource database" (http://www.mysql.com/)).   Of course, you can always setup a Solaris or OpenSolaris environment and download it (70-80MB) and then install/configure locally.   However, if you don't want to go through the hassle of setting up the System, OS, & database yourself, Sun has made it easy to get your hands on pre-configured bits in a couple of other efficient ways.

    Use the EZQual Virtual Lab.  Sun Partner's can get access to secure, no-cost, test environments (within Solaris containers/zones).  Here's the information directly from the program:

      [Read more...]
    Sun's Cloud Computing Portfolio
    Employee +0 Vote Up -0Vote Down

    Update: Sun has expanded its Cloud Computing portfolio with the recent acquisition of Qlayer, a cloud computing company that automates the deployment and management of both public and private clouds.  The Q-layer organization, based in Belgium, is now part of Sun's Cloud Computing business unit which develops and integrates cloud computing technologies, architectures and services.

    Cloud computing is about managing petascale data. Sun's server and storage systems can radically improve the data-intensive computing emerging in the cloud. Some clouds are closed platforms that lock you in. Sun's open source philosophy and Java principles form the core of a strategy that provides

      [Read more...]
    Sun's Cloud Computing Portfolio
    Employee +0 Vote Up -0Vote Down

    Update: Sun has expanded its Cloud Computing portfolio with the recent acquisition of Qlayer, a cloud computing company that automates the deployment and management of both public and private clouds.  The Q-layer organization, based in Belgium, is now part of Sun's Cloud Computing business unit which develops and integrates cloud computing technologies, architectures and services.

    Cloud computing is about managing petascale data. Sun's server and storage systems can radically improve the data-intensive computing emerging in the cloud. Some clouds are closed platforms that lock you in. Sun's open source philosophy and Java principles form the core of a strategy that provides

      [Read more...]
    Sun's Cloud Computing Portfolio
    Employee +0 Vote Up -0Vote Down

    Update: Sun has expanded its Cloud Computing portfolio with the recent acquisition of Qlayer, a cloud computing company that automates the deployment and management of both public and private clouds.  The Q-layer organization, based in Belgium, is now part of Sun's Cloud Computing business unit which develops and integrates cloud computing technologies, architectures and services.

    Cloud computing is about managing petascale data. Sun's server and storage systems can radically improve the data-intensive computing emerging in the cloud. Some clouds are closed platforms that lock you in. Sun's open source philosophy and Java principles form the core of a strategy that provides

      [Read more...]
    Virtualizing MySQL
    +0 Vote Up -0Vote Down

    I had so much to say in response to a recent post asking about virtualization from Jennifer Glore that I realized it was long enough to be a blog post.

    It really depends on what you’re looking to do. Many companies don’t have the money and staff to have an in-house data center with proper power and network redundancy; others don’t want the depreciation associated with owning computer hardware (even if they leased space in a data center, they’d have to buy equipment to put in it).

    Some reasons to virtualize:
    1) you need a fresh machine and cannot wait to order a new one or re-purpose an older one.
    2) your need for machines/services fluctuates (and again, re-purposing takes time). This need can be as broad

      [Read more...]
    Success with OpenSolaris + ZFS + MySQL in production!
    +0 Vote Up -0Vote Down

    Pimp My Drive by Richard and Barb

    There’s remarkably little information online about using MySQL on ZFS, successfully or not, so I did what any enterprising geek would do: Built a box, threw some data on it, and tossed it into production to see if it would sink or swim.

    I’m a Linux geek, have been since 1993 (Slackware!). All of SmugMug’s datacenters (and

      [Read more...]
    Your data and the cloud
    +0 Vote Up -0Vote Down

    I will be speaking on July 29th in New York at an Entrepreneurs Forum on A Free Panel on Cloud Computing. With a number of experts including Hank Williams of KloudShare, Mike Nolet of AppNexus, and Hans Zaunere of New York PHP fame is should be a great event.

    The focus of my presentation will be on “Extending existing applications to leverage the cloud” where I will be discussing both the advantages of the cloud, and the complexities and issues that you will encounter such as data management, data consistency, loss of control, security and latency for example.

    Using traditional MySQL based applications I’ll be providing an approach that can lead to your application gaining greater power of cloud computing.


    About the Author

      [Read more...]
    Setting up on EC2
    +0 Vote Up -0Vote Down

    Thanks to my friend Dustin, and his EC2 demo using Elasticfox Firefox Extension for Amazon EC2 I got an EC2 image setup. With other references Link 1,Link 2,Link 3 I was also able to create my own AMI.

    Some notes specific for my configuration.

    Pre-config ElasticFox key for launching directly from ElasticFox SSH connections.

    mkdir ~/ec2-keys
    mv ~/Downloads/elasticfox.pem ~/ec2-keys/id_elasticfox
    chmod 600 ~/ec2-keys/id_elasticfox
    chmod 700 ~/ec2-keys/
    ssh -i
      [Read more...]
    OpenSolaris (and GlassFish and MySQL) on Amazon EC2
    Employee_Team +0 Vote Up -0Vote Down

    You may have seen Jonathan's note on OpenSolaris, MySQL and GlassFish being available on Amazon EC2.

    Details on the OpenSolaris portion are available at the EC2 Blog (Welcome, Launch and New Limits), at Dileep's Blog

      [Read more...]
    Don McAskill - People I met at MySQL Conference
    +0 Vote Up -0Vote Down
    "The two metrics that are most important to me are first customer satisfaction and second growth." - Don McAskill

    Today, I noticed Don is featured on Sun's customer success stories page:


    Don McAskill is the CEO and Chief Geek of Smugmug, a photo and now hi-def video (using H.264) sharing site with a successful business model behind it.

    I initially met Don last year at the MySQL Conference when my then boss told me that he is interested in meeting him. That was my introduction to Smugmug. I was impressed by SmugMug's presentation of photos and the care they took to make your photos and galleries look awesome.

    This year, as a member of Smugmug, me and my wife got to interact with Don on a personal level.

    We had










      [Read more...]
    MySQL Conf08 - Hangin' with Brian Aker
    +0 Vote Up -0Vote Down

    Here is number 5 in my series of six podcasts from last week's MySQL conference and expo.

    Just after lunch on Tuesday, I was able to corner Brian Aker, former CTO of MySQL, introduce myself and ask him if he was up for a podcast.  Without any convincing or arm twisting he happily agreed. :)

    My interview with Brian (9:18)  Listen (Mp3)   Listen (ogg)



      [Read more...]
    MySQL Conference Day 2 Thoughts
    +0 Vote Up -0Vote Down

    Keynotes

    The keynote was kick started by Marten Mickos.  If you've never met Marten, he is, on a personal note, one of the greatest CEOs I've ever met.  The keynotes were especially interesting for me because it was the first time I've had the opportunity to listen to Jonathan Schwartz, the CEO of Sun Microsystems.  Jonathan seems like a great guy who gives the impression he "gets it".

    The last keynote was by Werner Vogels of Amazon.  His talk covered Amazon's growth and the new services they offer including EC2.  He announced that EC2 now supports persistent storage, which is a huge improvement, but doesn't quite solve all of the problems.

    Testing PHP/MySQL Applications with PHPUnit/DbUnit

    I've never been big into testing, but I'm trying to

      [Read more...]
    Is EC2 useful as a database server
    +0 Vote Up -0Vote Down
    Plenty of people have been excited by the prospect of Amazon EC2 and the ability to scale out your databases as load increases from your original configuration. I noticed Morgan Tocker and Carl Mercier are going to be presenting on this topic at the upcoming MySQL ConferenceHowever almost immediately people are worried about the lack of persistent of data across instance terminations.In a sense
    Sysbench fileio vs EC2 Part 1
    +0 Vote Up -0Vote Down
    Overview:Peter Zaitsev's recent article about Evaluating IO subsystem performance for MySQL spurred my interest in doing something similar on EC2.I have covered running sysbench against MySQL on EC2 however not specifically used sysbench to test IO. Rather I had used bonnie++ and iozone to do that.I don't have a lot of respect for the EC2 small instance. Whilst it was reasonable in the middle of
    MySQL Master-Master replication table sync
    +0 Vote Up -0Vote Down
    I saw a post by Baron mentioning that his tool maatkit is best for handling situations where a master-master replication setup has got out of sync. If you think Baron was blowing his own trumpet he has good reason to. I have used his mk-archiver tool as part of the Maatkit to make the problem of archiving and purging data much easier. This was much easier than rolling my own solution. Anyhow. I
    Bonnie++ Benchmark on EC2
    +0 Vote Up -0Vote Down
    I'm getting in early, I had just started building my benchmarks for my talk on Amazon EC2 at the MySQL Conference and Expo next year when I discovered exactly why they say you need to run a test more than once; results can be completely unpredictable. Take for example the Sequential I/O performance on EC2, versus my home machine:



    The first 3 tests were a 36G Seagate Raptor 10k RPM, a 160G Seagate SATA2 7200RPM, and a 320G Seagate IDE 7200RPM disk, running in the same machine I had at home. The last three were Amazon EC2 images. A few observations from these results:
    • The char write test seems to max out my CPU (not on graph - see raw data), so that probably explains why it's consistent across all disks.
    • My home machine is almost dead on consistent, whereas Amazon EC2





      [Read more...]
    What is the Next Big Thing? (longish)
    +0 Vote Up -0Vote Down

    In a decade, on-demand virtualized utility computing will be an invisible utility, part of the vital infrastructure of the technological economy.

    People will mostly have forgotten what an enormous pain in the ass provisioning computation was today. Today, we don't truly feel that pain, because it seems "normal", everyone has to suffer it together.

    The situation right now is, if you have a delivery van, you have to make your own gasoline. And you have to hire and pay for your own mechanics. Seems stupid, doesn't it? It's amazing that there are any delivery vans at all …

    Think of the internet itself, what it did to telecoms.

    Twenty-five years ago, if you wanted a high speed data connection to a computer in San Francisco, it was a pain. You'd have to come up with a pile of money, and wait a couple of months, at best. Hardware would be dedicated and provisioned, and

      [Read more...]
    Previous 30 Newer Entries Showing entries 31 to 60 of 61 Next 1 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.