Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 72 Next 30 Older Entries

Displaying posts with tag: storage (reset)

Storage caching options in Linux 3.9 kernel
+0 Vote Up -0Vote Down

dm-cache is (albeit still classified “experimental”) is in the just released Linux 3.9 kernel. It deals with generic block devices and uses the device mapper framework. While there have been a few other similar tools flying around, since this one has been adopted into the kernel it looks like this will be the one that you’ll be seeing the most in to the future. It saves sysadmins the hassle of compiling extra stuff for a system.

A typical use is for an SSD to cache a HDD. Similar to a battery backed RAID controller, the objective is to insulate the application from latency caused by the mechanical device, the most laggy part of which is seek time (measured in milliseconds). Giventhe  relatively high storage capacity of an SSD (in the hundreds of GBs), this allows you to mostly disregard the mechanical latency for writes and that’s very useful for

  [Read more...]
The “Big Data” buzzword finally gets a real definition
+0 Vote Up -0Vote Down

We’ve all heard the term “Big Data” thrown around a fair amount in the last several years ever since the rise of Hadoop and other distributed storage methods. But defining “Big Data” has always been a subjective term that hinges on perspective; what one engineer considers big can be vastly different than another’s.

However, there’s finally a definite description that says Big Data no matter what perspective you operate from: “That facility by my calculations that I submitted to the court for the Electronic Frontiers Foundation against NSA would hold on the order of 5 zettabytes of data. Just that current storage capacity is being advertised on the web that you can buy. And that’s not talking about what they have in the near future.” You can read more about the facility and its purpose here:

  [Read more...]
The blog was down yesterday
+0 Vote Up -0Vote Down

The brief outage was due to a scheduled move of the servers to a separate rack and subnet dedicated to our work with the Center for Information Assurance & Cybersecurity (ciac) at the University of Washington Bothell (uwb), and a11y.com

I am currently exercising the new (to us) equipment and hope to winnow the less than awesome equipment over the next quarter. I spent the last six months finding the best in breed of the surplussed DL385 and DL380 chassis we (work) were going to have recycled. The team and I were able to find enough equipment to bring up one of each with eight and six gigs of memory, respectively. These will make excellent hypervisors for provisioning embedded instances of Slackware, Fedora, RHEL, CentOS, Debian, FreeBSD, OpenSolaris, OpenIndiana, FreeDOS, etc.

When I initially configured this xen paravirt environment, I failed to plan for integration with libvirt, so I am

  [Read more...]
Amazon EC2 now powered by high performance storage (benchmark)
+3 Vote Up -0Vote Down

A while ago I had a discussion with someone about the future of server infrastructure. Among other things, we were wondering whether companies will continue to run on dedicated servers or if eventually everyone just ends up in a Cloud environment. During the discussion I raised a point that in principle Cloud is a great idea that will keep attracting more and more people, but it is missing one important piece that stops many from using it – a high performance storage. Apparently, this has just changed.

Yesterday I received an e-mail announcing a new EC2 instance type – hi1.4xlarge. It features 16 logical CPUs (35 ECUs), 60GB of RAM, and… two 1TB SSD-based disk volumes! These are great specs that should work for nearly any database. Even assuming someone has a MySQL database larger than 2TB, not all tables will

  [Read more...]
MySQL on S3: performance with storage located across the continent
+2 Vote Up -0Vote Down

Can OLTP database workloads use Amazon S3 as primary storage? Now they can, thanks to the Cloud Storage Engine (ClouSE), but the question is: how fast?


To answer the question about performance, we decided to run db_STRESS benchmark on a MySQL database in Amazon EC2.  We compared 3 configurations:

  • “Across the street storage”: ClouSE with data stored in S3 in the same region (US Standard)
  • “Across

  [Read more...]
Running out of disk space on MySQL partition? A quick rescue.
+2 Vote Up -0Vote Down

No space left on device – this can happen to anyone. Sooner or later you may face the situation where a database either has already or is only minutes away from running out of disk space. What many people do in such cases, they just start looking for semi-random things to remove – perhaps a backup, a few older log files, or pretty much anything that seems redundant. However this means acting under a lot of stress and without much thinking, so it would be great if there was a possibility to avoid that. Often there is. Or what if there isn’t anything to remove?

While xfs is usually the recommended filesystem for a MySQL data partition on Linux, the extended filesystem family continues to be very popular as it is used as default in all major Linux distributions. There is a feature

  [Read more...]
Now available: Slides from Percona Live and Linuxcon Europe
+0 Vote Up -1Vote Down

The slides from last week’s talks I (co-)presented at Percona Live and Linuxcon Europe are now available from our web site.

  [Read more...]
Got open source cloud storage? Red Hat buys Gluster
+0 Vote Up -0Vote Down

Red Hat’s $136m acquisition of open source storage vendor Gluster marks Red Hat’s biggest buy since JBoss and starts the fourth quarter with a very intersting deal. The acquisition is definitely good for Red Hat since it bolsters its Cloud Forms IaaS and OpenShift PaaS technology and strategy with storage, which is often the starting point for enterprise and service provider cloud computing deployments. The acquisition also gives Red Hat another weapon in its fight against VMware, Microsoft and others, including OpenStack, of which Gluster is a member (more on that further down). The deal is also good for Gluster given the sizeable price Red Hat is paying for the provider of open source, software-based, scale-out storage for unstructured data and also as validation of both open source and software in

  [Read more...]
zfs FileSystem and MySQL
+2 Vote Up -2Vote Down



ZFS is a new kind of 128-bit file system that provides simple administration, transactional semantics, end-to-end data integrity, and immense scalability. ZFS is not an incremental improvement to existing technology; it is a fundamentally new approach to data management. ZFS was first introduced in Solaris in 2004 and it is a default filesystem in OpenSolaris, but Linux ports are underway, Apple is shipping it in OS X 10.5 Leopard with limited zfs capability ( Apple shutdown this project afterward due to some known reason), and it will be included in FreeBSD 7.


  [Read more...]
RE: A bit on SANs and system dependencies
+0 Vote Up -0Vote Down
This is a reply on A bit on SANs and system dependencies by Eric Bergen.

Lets first start by making a difference between entry level, midrange and high-end SAN's.

Entry level:
This is basically a bunch of disks with a network connection. The Oracle/Sun/StorageTek 2540 is an example for this category. This storage is aimed at lowcost shared storage.

Midrange:
This kind of storage generally features replication and more disks than entry level. HP EVA is what comes to mind for this category. This storage is aimed at a good price/performance.

High-End:
This is mainframe style storage which is made usable for open systems. Hitachi Data Systems VSP is a good example of this category. This category is for










  [Read more...]
OpenDBCamp: Information Lifecycle Architecture
+2 Vote Up -2Vote Down
The Open DB Camp in Sardinia 2011 has had a number of sessions on varying topics. Topics range from MySQL over MongoDB to replication and High Availability.

I decided to tap into the database expert resources present here at Sardegna Ricerche by discussing a non-database issue, where one can expert database experts to have insights beyond those of end users. And they did.

The topic was the particular case of information overload many of us suffer from on our hard disks: Too many files, too hard to find.
  • How do we find the bank statement from April 2007 from the





  [Read more...]
Roll Your Own Cloud (from linux.conf.au 2011)
+0 Vote Up -0Vote Down

At linux.conf.au 2011 in Brisbane, I had the honor of co-presenting a talk on Rolling Your Own Cloud with SUSE’s Tim Serong. Take a look!



  [Read more...]
Tech Messages | 2011-01-15
+0 Vote Up -0Vote Down

A special extended edition of Tech Messages for 2011-01-12 through 2011-01-15:

  [Read more...]
DRBD != fsck != DIX
+1 Vote Up -0Vote Down

Every once in a while, we hear of users with corruption in a file system that sits on top of DRBD. That may be easy or tricky to resolve. If you’re lucky, a simple fsck will resolve the corruption. If you’re not quite that lucky, you may have to get out your backups.

But that’s typically not DRBD’s fault. Typically not at all, not in the least bit. DRBD is a block device, and as such it has no idea what rests on top of it. It has no concept of a filesystem, let alone its integrity. That of course is true for any other block device as well. If you have, say, RAID-1, and something corrupts the file system on top of it, then of course that corruption will be happily replicated across both component devices. DRBD is no different, except that its component devices are stored across distinct physical nodes.

And even if everything about your filesystem

  [Read more...]
LCA Miniconf Call for Papers: Data Storage: Databases, Filesystems, Cloud Storage, SQL and NoSQL
+3 Vote Up -0Vote Down

This miniconf aims to cover many of the current methods of data storage and retrieval and attempt to bring order to the universe. We’re aiming to cover what various systems do, what the latest developments are and what you should use for various applications.

We aim for talks from developers of and developers using the software in question.

Aiming for some combination of: PostgreSQL, Drizzle, MySQL, XFS, ext[34], Swift (open source cloud storage, part of OpenStack), memcached, TokyoCabinet, TDB/CTDB, CouchDB, MongoDB, Cassandra, HBase….. and more!

Call for Papers open NOW (Until 22nd October).

ScaleDB Cache Accelerator Server (CAS): A Game Changer for Clustered Databases
+0 Vote Up -0Vote Down
ScaleDB and Oracle RAC are both clustered databases that use a shared-disk architecture. As I have mentioned previously, they both actually share data via a shared cache, so it might be more appropriate to call them shared-cache databases.

Whether it is called shared-disk or shared-cache, these databases must orchestrate the sharing of a single set of data amongst multiple nodes. This introduces two challenges: the physical sharing of the data and the logical sharing of the data.

Physical Sharing:
Raw storage is meant to work on a 1:1 basis with a single server. In order to share that data amongst multiple servers, you need either a Network File System (NFS), which shares whole files, or a Cluster File System (CFS), which shares data blocks.






  [Read more...]
No, DRBD doesn’t magically make your application crash safe
+4 Vote Up -0Vote Down

It is a common misconception that DRBD (or any block-level data replication) solution can magically make an application crash-safe that intrinsically isn’t. Baron highlights that misconception in a recent blog post.

I want to reiterate and stress that point here: if your application can’t reliably survive a node crash, it won’t successfully fail over on a replicated (or shared, for that matter) data device. But if it can, and DRBD is replicating synchronously, then DRBD won’t break it. In other words: try pulling the power plug on your machine while your app is running, and power back on. If your application recovers to a consistent state, you’re

  [Read more...]
Shared Cache Tier & Storage Flexibility
+1 Vote Up -0Vote Down
Any time you can get two for the price of one (a “2Fer”), you’re ahead of the game. By implementing our shared cache as a separate tier, you get (1) improved performance and (2) storage flexibility…a 2Fer.

What do I mean by storage flexibility? It means you can use enterprise storage, cloud storage or PC-based storage. Other shared-disk cluster databases require high-end enterprise storage like a NAS or SAN. This requirement was driven by the need for:

1. High-performance storage
2. Highly available storage
3. Multi-attach, or sharing data from a single volume of LUN across multiple nodes in the cluster.

Quite simply, you won’t see other shared-disk clustering databases using cloud storage or PC-based storage. However, the vast majority of MySQL users rely on PC-based storage, and most are not willing to pay the







  [Read more...]
BlitzDB Crash Safety and Auto Recovery
+0 Vote Up -0Vote Down

Crash Safety is a big deal in the database league. Lack of durability can lead to all sorts of terrible things upon a catastrophic event. Many projects, especially in the so called NoSQL world compromises crash safety in return for higher QPS. The argument there is that the availability of the overall system should be accomplished by replication since a database server can’t be rescued if the physical disk breaks. I happen to agree with this philosophy but I am also aware that this isn’t a correct answer for everyone. So, what will I do with BlitzDB?

Several relational database hackers have pointed out that BlitzDB isn’t any safer than MyISAM since it doesn’t guarantee crash safety. This is currently true but I plan on making BlitzDB much safer than MyISAM by providing following features.

  • Auto Recovery Routine (startup option)
  • Tokyo
  •   [Read more...]
    LinuxTag presentation now available for download
    +0 Vote Up -0Vote Down

    A live recording of my LinuxTag 2010 presentation entitled Storage Done Right: Building a Resilient, Distributed, Highly Available Open Source iSCSI SAN is now available from our web site. If you want to find out how to build a complete SAN from 100% open source, do take a look!

    I do apologize for the less-than-optimal sound quality. I did the recording myself with my laptop mike, so unfortunately there’s quite a bit of clipping in the audio track. I hope my ramblings are still somewhat audible.



      [Read more...]
    Speaking at LinuxTag 2010 this Friday
    +0 Vote Up -0Vote Down

    Just as a quick reminder, as LinuxTag 2010 in Berlin is going into full swing: I am speaking on highly available, distributed, open source storage this Friday in room Europa I at 4pm CEST.

    If you’re in Berlin, drop by! Admission is free.



      [Read more...]
    See you at LinuxTag 2010
    +0 Vote Up -0Vote Down

    I just got word that a talk about iSCSI High Availability with Pacemaker and DRBD, which I submitted for LinuxTag 2010, has been accepted by the selection committee.

    The presentation is entitled Storage Done Right: Building a Resilient, Distributed, Highly Available Open Source iSCSI SAN and I will talk in the Storage track, in English. Questions in German, of course, won’t be a problem.

    As the conference organizers have asked speakers not to publicize the temporary schedule (for obvious reasons — hey it’s temporary), I can’t give the exact time and location yet. But if you want to hear about how you can replace your six-figure SAN with something much more open, much less locked-in and much less

      [Read more...]
    Berkeley DB now supports SQL (again)
    Employee +5 Vote Up -0Vote Down

    Berkeley DB (BDB) is undoubtedly the workhorse among the opensource embedded database engines. It started as a university project in the mid-eighties and was further developed by Sleepycat Software, until it got acquired by Oracle in February 2006.

    I had the impression that BDB had lost a lot of its popularity among opensource developers to SQLite in recent times, which has evolved into becoming the default choice for developers looking for an embedded data store. I'd assume primarily because the code is not released under any particular license, but put in the public domain

      [Read more...]
    MySQL University: MySQL Column Databases
    Employee +2 Vote Up -0Vote Down

    This Thursday (March 4th, 15:00 UTC - slightly later than usual), Robin Schumacher will present MySQL Column Databases. If you're doing Data Warehouse with your databases this is a must-attend, but it's also interesting to learn what typical other scenarios there are for using column-based storage engines, and how column databases work in the first place.

    For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to. (Dimdim is the conferencing system


      [Read more...]
    MySQL University: MySQL Column Databases
    Employee +0 Vote Up -0Vote Down

    This Thursday (March 4th, 15:00 UTC - slightly later than usual), Robin Schumacher will present MySQL Column Databases. If you're doing Data Warehouse with your databases this is a must-attend, but it's also interesting to learn what typical other scenarios there are for using column-based storage engines, and how column databases work in the first place.

    For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to. (Dimdim is the conferencing


      [Read more...]
    MySQL University: MySQL Column Databases
    Employee +0 Vote Up -0Vote Down

    This Thursday (March 4th, 15:00 UTC - slightly later than usual), Robin Schumacher will present MySQL Column Databases. If you're doing Data Warehouse with your databases this is a must-attend, but it's also interesting to learn what typical other scenarios there are for using column-based storage engines, and how column databases work in the first place.

    For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to. (Dimdim is the conferencing


      [Read more...]
    CAOS Theory Podcast 2010.02.05
    +0 Vote Up -0Vote Down

    Topics for this podcast:

    *Matt Asay moves from Alfresco to Canonical
    *GPL fade fuels heated discussion
    *Apple’s iPad and its enterprise and open source impact
    *Open source in data warehousing and storage
    *Our perspective on Oracle’s plans for Sun open source

    iTunes or direct download (32:50, 9.2 MB)

    Notes on HEAP/MyISAM Index Key Handling on WRITE
    +0 Vote Up -0Vote Down

    Disclaimer: This post is based on HEAP/MyISAM’s sourcecode in Drizzle.

    Here are my brief notes on investigating how index keys are generated in HEAP and MyISAM. I lurked through these because I’ve started preparing for decent index support in BlitzDB. I also wrote this to assist my biological memory for later grepping (I have terrible memory for names). I’m only going to cover key generation on write in this post. Otherwise this post is going to be massive.

    HEAP Engine

    The index structure of HEAP can be either BTREE or HASH (in MySQL doc terms). Like other engines HEAP has a structure for keeping Key definition (parts, type, logic and etc). This structure is called HP_KEYDEF and it contains function pointers for write, delete, and getting the length of the key. These function pointers are assigned to at table creation or when the

      [Read more...]
    TEXT vs. VARCHAR
    +7 Vote Up -2Vote Down

    On first glance, it looks like TEXT and VARCHAR can store the same information. However, there are fundamental differences between the way TEXT fields and VARCHAR fields work, which are important to take into consideration.

    Standard
    VARCHAR is actually part of the ISO SQL:2003 standard; The TEXT data types, including TINYTEXT, are non-standard.

    Storage
    TEXT data types are stored as separate objects from the tables and result sets that contain them. This storage is transparent — there is no difference in how a query involving a TEXT field is written versus one involving a VARCHAR field. Since TEXT is not stored as part of a row, retrieval of TEXT fields requires extra [edited 1/22] memory overhead.


    Maximum VARCHAR length
    The maximum row length of a VARCHAR is restricted by the



      [Read more...]
    MySQL University: The Spider Storage Engine
    Employee +2 Vote Up -0Vote Down

    This Thursday (November 26th, 14:00 UTC), Giuseppe Maxia will present the Spider Storage Engine. This session was originally scheduled for October 15th but had to be postponed for technical reasons.

    Here's from the abstract: Everybody needs sharding. Which is not easy to maintain. Being tied to the application layer, sharding is hard to export and to interact with. The Spider storage engine, a plugin for MySQL 5.1 and later, solves the problem in a transparent way. It is an extension of partitioning. Using this engine, the user can deal transparently with multiple


      [Read more...]
    Showing entries 1 to 30 of 72 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.