Showing entries 31 to 40 of 73
« 10 Newer Entries | 10 Older Entries »
Displaying posts with tag: ec2 (reset)
Scaling-out OLTP load on Amazon EC2 revisited.

It's been long known that Galera optimistic replication and enterprise-size databases are a match made in heaven. Today we're going to get a little closer to testing this statement.
We'll have look at how Galera can scale out Sysbench OLTP complex 60 million rows workload in EC2. This is a first proper benchmark for 0.8 series and also the first benchmark of MariaDB/Galera port, so I'll start modest, just to see how it goes. I chose m1.large instances with 7.8Gb of RAM for server nodes and c1.xlarge instance for a client - I don't want the client to be a bottleneck.

For comparison I have also measured performance of a stock standalone MariaDB 5.1.55 server. I used the standard my.cnf that comes with MariaDB Debian package with the following alterations:

max_connections=1024
innodb_buffer_pool_size=6G
innodb_log_file_size=512M

Galera nodes also add

[Read more]
Hey, it's time for the MySQL User Conference again! Come see me!

And I mean that, come see me, say hello, buy me a beer (extra points!). Or even more so, come see my session at 10:50 on tuesday morning April 12. I'll be speaking on how to manage large datasets in an Amazon EC2 environment, and this is largely based on my experiences at doing just that at my new job (or new, I've been doing it for more that 6 months now) as Database Architect at Recorded Future.

This will not be an incredibly technical presentation, in terms of showing actual code and things. Rather, I will look at some of the issues when running in an EC2 environment, and how we manage it here at Recorded Future. Also, I will present a bit of how our architecture works, which is more relevant that one may thinks, as we have Cloud based architectures on mind all the time, all our development, testing and productions servers run in the cloud.

Anyway, this is …

[Read more]
EC2 - The E is for Elastic

So, you are thinking about Cloud Computing? Is it a fad, along the lines of SOA, OOP, NoSQL, ORDBMS or is it a new paradigm when it comes to infrastructure? (not that a fad is bad, it's just that a fad, in my mind, is something that is grossly overblown in proportion. OOP is a good thing, but tell you what, OO-talibans out there, despite what you may think, OOP will not create peace in the middle east (if it did, I'd embrace it right now)).

But all that aside, what is in the Cloud, really? And from a technical standpoint, it seems simple enough: Your servers running across a number of virtual machines, with virtual disks and what have you not, where you pay for resource use and you share the environment with a bunch of other users. And that really is not that complicated. And from a pure technical view, that is it, sort of, but there is more to it than that, because when you come to run your stuff in a cloud, you realize that things …

[Read more]
Amazon moves into PaaS with Elastic Beanstalk, Java as 1st class citizen

Amazon's EC2 and its sister S3 service have been indisputable leaders in IaaS for a long while now and GlassFish and more generally J2EE/JavaEE took advantage of it starting in 2008 (see here and here), with documented how-to's and significant production references. …

[Read more]
Defining clouds, web services, and other remote computing

Series

What are the chances for a free software cloud?

[Read more]
Amazon now accepts hard drives for EC2 data transfer

I guess they got tired of people sending angry emails about data transfer fees:

“Amazon provides an online calculator to help customers decide whether it makes financial sense to ship data via mail rather than uploading over the Internet. You plug in the number of terabytes, devices, average file size, return shipping information and other factors, and find out how much the data transfer would cost via mail compared to standard Internet uploads.

For example, transferring data from a single device containing 2TB would require 26 hours of data loading time and cost $144.74. Uploading the same amount of data over the Internet would cost $204.80. The calculator does not show how long the Internet transfer would take.”

[Read more]
Piper Jaffray on the Cloud

Piper Jaffray has published a 300+ page study on the cloud computing industry based on a recent survey undertaken of 100 CIOs. Bottom line, cloud computing is expected to grow significantly over the next five years. 

    Survey respondents expect the mix of cloud computing to escalate strongly to 13.5% in five years. This equates to a five-year CAGR of 19.2%, or 23.9% when we also incorporate IDC’s forecast that total software budgets will grow 4.7% annually. In other words, software spending will grow gradually in the next five years, but the mix of spend allocated to cloud-based applications will likely surge rapidly. Another way to think about the data is that the Cloud Computing market is expected to grow five times as fast as the broader software market: 23.9% vs. 4.7%.

If anything, I think the prediction is conservative and the impact could be much larger in magnitude when mainstream …

[Read more]
Looks like Amazon is listening

Just got the following in my email this morning. I sure wish they had done this earlier. “Free Inbound Data Transfer (until June 30, 2010) Data Transfer into AWS will be free of charge from now through June 30, 2010, making it even easier for customers to get their data into AWS. This applies to [...]

Dissection of EC2 / EBS volume

So during preparation of XtraDB template for EC2 I wanted to understand what IO characteristics we can expect from EBS volume ( I am speaking about single volume, not RAID as in my previous post). Yasufumi did some benchmarks and pointed me on interesting behavior, there seems several level of caching on EBS volume.

Let me show you. I did sysbench random read IO benchmark on files with size from 256M to 5GB with step 256M. And, as Morgan pointed me, I previously made first write, to avoid first-write penalty:

dd if=/dev/zero of=/dev/sdk bs=1M

for reference script is:

PLAIN TEXT CODE:

  1. #!/bin/sh
  2. set -u
  3. set -x
[Read more]
EC2/EBS single and RAID volumes IO benchmark

During preparation of Percona-XtraDB template to run in RightScale environment, I noticed that IO performance on EBS volume in EC2 cloud is not quite perfect. So I have spent some time benchmarking volumes. Interesting part with EBS volumes is that you see it as device in your OS, so you can easily make software RAID from several volumes.

So I created 4 volumes ( I used m.large instance), and made:

RAID0 on 2 volumes as:
mdadm -C /dev/md0 --chunk=256 -n 2 -l 0 /dev/sdj /dev/sdk

RAID0 on 4 volumes as:
mdadm -C /dev/md0 --chunk=256 -n 4 -l 0 /dev/sdj /dev/sdk /dev/sdl /dev/sdm

RAID5 on 3 volumes as:
mdadm -C /dev/md0 --chunk=256 -n 3 -l 5 /dev/sdj /dev/sdk /dev/sdl

RAID10 on 4 volumes in two steps:

mdadm -v --create /dev/md0 --chunk=256 --level=raid1 --raid-devices=2 …

[Read more]
Showing entries 31 to 40 of 73
« 10 Newer Entries | 10 Older Entries »