Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Previous 30 Newer Entries Showing entries 31 to 60 of 90 Next 30 Older Entries

Displaying posts with tag: partitioning (reset)

MySQL: Partition-wise backups with mysqldump
+4 Vote Up -0Vote Down
To whom it may concern,

in response to a query from André Simões (also known as ITXpander), I slapped together a MySQL script that outputs mysqldump commands for backing up individual partitions of the tables in the current schema. The script is maintained as a snippet at MySQL Forge.

How it works

The script works by querying the

  [Read more...]
MySQL user group meeting in Dubai, January 13th
+5 Vote Up -1Vote Down



On January 13th I will be in Dubai, UAE, on my way to Wellington, New Zealand.
I will be at the MySQL User Group, hosted at Sun Microsystems offices at 6pm.
I will talk about boosting performance with MySQL 5.1 partitions, covering the recent 5.5. additions.
The meeting is open to all. If you want to attend, please register at the meetup page.
Holiday gift - A deep look at MySQL 5.5 partitioning enhancements
+13 Vote Up -0Vote Down



Half a day into my vacation, I managed to finish an article on a topic that has been intriguing me for a while.
Since several colleagues were baffled by the semantics of the new enhancements of MySQL 5.5 partitions, after talking at length with the creator and the author of the manual pages, I produced this article: A deep look at MySQL 5.5 partitioning enhancements.
Happy holidays!

UPDATE This matter was more tricky than it appeared at first sight. As Bug#49861 shows, several MySQL engineers were






  [Read more...]
Getting started with MySQL 5.5
+9 Vote Up -0Vote Down



Some time go, we announced a new release model for MySQL. As all new things, it had some initial hiccups (with MySQL 5.4 we were still getting acquainted with the new model), but now it seems to be in full swing.
By the time you read these lines, MySQL 5.5 will be available. If the mirrors aren't seeded yet, the impatient can compile and use the new version from the launchpad source tree..

Overview

What's this new release anyway? I'll leave it




  [Read more...]
Improvement of MySQL partitioning included in MySQL's next milestone release tree
Employee +4 Vote Up -0Vote Down
It was quite some time since I last blogged. It's not due to
inactivity. For those of you that have followed my blog might
have seen earlier blog posts about a new partitioning feature.

This new partitioning I first blogged about in July 2006 and
that blog is still the 3rd most popular blog of my blogs, even
when looking at the last months views. The work on this started
out in 2005 and so it's nice to now get it in a state where it's
quality is ready for more heavy testing. For those interested
in partitioning I think this feature will enlarge the number of
cases where partitioning is applicable. It's now possible to
partitioning on many more field types and also on multiple fields
in an efficient manner.

This feature described by WL#3352













  [Read more...]
Calpont opens up: InfiniDB Open Source Analytical Database (based on MySQL)
+9 Vote Up -0Vote Down
Open source business intelligence and data warehousing are on the rise!

If you kept up with the MySQL Performance Blog, you might have noticed a number of posts comparing the open source analytical databases Infobright, LucidDB, and MonetDB. LucidDB

  [Read more...]
Spider and vertical partition engines with new goodies
+4 Vote Up -0Vote Down



The Spider storage engine should be already known to the community. Its version 2.5 has recently been released, with new features, the most important of which is that you can execute remote SQL statements in the backend servers. The method is quite simple. Together with Spider, you also get an UDF that executes SQL code in a remote server. You send a query with parameters saying how to connect to the server, and check the result (1 for success, 0 for failure). If the SQL involves a SELECT, the result can be sent to a temporary table. Simple and effective.

In addition to the Spider engine, Kentoku




  [Read more...]
Partitioning with non integer values using triggers
+6 Vote Up -1Vote Down
Looking at Bug#47310, which is a feature request that I hear frequently when I talk about partitions, I wrote a comment, suggesting triggers to work around the limitation.
The reason for the limitation is that allowing arbitrary functions for partitioning was too complex and it was provoking crashes and other unpleasant side effects (see the discussion under bug#18198).
But if you use a trigger, the resulting column is a plain integer, and many of the side effects disappear. The drawback is that you need to add a column to your table, and you need to use that column when searching for data. With that in mind, you can implement the workaround quite easily.

USE test;
DROP TABLE IF EXISTS users;

CREATE TABLE users (








  [Read more...]
Enterprise Software as a Service (SaaS) and Partitioning
+0 Vote Up -0Vote Down

 

Partitioning, new with MySQL 5.1, has complicated interactions with queries and indexes.  If one isn’t careful it is easy to degrade performance.   For example, select queries that go with that grain (queries where partition elimination occurs) can be much quicker, but select queries that go against that grain can be much slower.   Queries that go against the grain must query each partition, so for a table with 12 partitions,  one query against that  table can result in 12 queries, one against each of the partitions.  An example of this would be a query against a month partitioned table that is looking to see how much activity a product had in the past 12 months. 

The ideal partitioning scheme would be a system where all queries only needs to access data from one

  [Read more...]
partitioning + auto_increment is buggy!
Employee +0 Vote Up -0Vote Down
Folks, I just want to warn you how buggy partitioned tables with negative values are. Never use negative values for InnoDB auto_increment columns!!!!

There are more bugs than I listed here..

  [Read more...]
MySQL Sandbox and Spider at FrOSCon and OpenSQLCamp
+1 Vote Up -0Vote Down



FrOSCon and the OpenSQLCamp are about to start.
I am packing for Sankt Augustin, where I will attend the fourth edition of FrOSCon and the second OpenSQLCamp. I will have two sessions, Sharding for the masses, about the Spider storage engine and MySQL Sandbox 3, about one of my favorite tools.

The program is very rich. There will be several tracks in the main event and in the associated conferences. If you have any involvement or simply some





  [Read more...]
Why you don’t want to shard.
+6 Vote Up -1Vote Down

Note: This blog post is part 1 of 4 on building our training workshop.

The Percona training workshop will not cover sharding. If you follow our blog, you'll notice we don't talk much about the subject; in some cases it makes sense, but in many we've seen that it causes architectures to be prematurely complicated.

So let me state it: You don't want to shard.

Optimize everything else first, and then if performance still isn't good enough, it's time to take a very bitter medicine. The reason you need to shard basically

  [Read more...]
Partition by column_list ready for alpha testers
Employee +3 Vote Up -0Vote Down
I got time to spend on a really old worklog I completed coding
already october 2005. I blogged about it in July 2006 and
interestingly enough it's still the second most read blog
entry on my blog (probably related to search engines in some
way).

I have merged it with the azalea tree (this is an internal code
name for our development tree, name is likely to change). This
tree contains subquery optimisations, Batched join and some more
optimisations.

I have fixed a whole bunch of bugs that always shows up in early
code. The code quality is still alpha but at least you won't find
10 bugs per hour :)


Here
you can find the launch pad tree for this code.

There are two important additions

















  [Read more...]
Sharding for the masses: the spider storage engine
+5 Vote Up -0Vote Down


In my previous article about the Spider storage engine, I made some tests and I saw that the engine has potential. I did also identify some problems, which were promptly addressed by the author. I have looked at the latest version (0.12), and the results are very much encouraging.

Installing the Spider storage engine is not trivial but not extremely difficult either. My previous article has a



  [Read more...]
MySQL University: Boosting Performance With MySQL 5.1 Partitioning
Employee +3 Vote Up -0Vote Down

This Thursday (June 4th, 14:00 UTC), Giuseppe Maxia will give a MySQL University session on Boosting Performance With MySQL 5.1 Partitioning. Giuseppe is leading the Community team at MySQL and has done various MySQL University sessions before.

For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to. (Dimdim is the conferencing system


  [Read more...]
MySQL University - Boosting performance with partitions
+2 Vote Up -0Vote Down



Mark your calendars: A MySQL University session about Boosting performance with MySQL 5.1 will take place on Thursday, June 4th at 13:00 UTC ( 8am CDT (Central) / 9am EDT (Eastern) / 14:00 BST / 15:00 CET / 17:00 MDT (Moscow) / 18:30 IST (India))
The session will be conducted through DimDim, a system that allows you to follow the audio and visuals of a presentation from your browser, without any additional settings.

Attendance is free. Please follow the instructions given in the





  [Read more...]
Partitioning Presentation from MySQL Conference
+0 Vote Up -0Vote Down

It appears that my presentation isn’t available at the conference site. I’ve added it below.   Sorry for the delay. 

Without some context that I talked about at the conference the presentation may not make sense.

Partitioning creates a grain in the table.  Select queries that go with that grain can be quicker, at times much faster, but select queries that go against that grain can be slower, at times much slower. 

An example is a table that is partitioned by date and has an orderId primary key.  Queries that select data by date will either be almost as fast (partitioning can add a slight overhead) as the non-partitioned table to much faster.   But queries that don’t query by date will have to query all the partitions.  A table partitioned 12 times, one partition for

  [Read more...]
Test driving the Spider storage engine - sharding for the masses
+7 Vote Up -0Vote Down


At the MySQL Conference 2009 I attended a session about the Spider storage engine, an engine with built-in sharding features.
The talk was notable for the speaker wearing a spiderman costume, and for some language barrier that made the talk less enjoyable than it should be. That's a pity, because the engine is very intriguing, and deserves some exploration.

What is the Spider engine, then? In short, it's an extension to the partitioning engine with the ability of connecting to remote servers. Basically, partitions + federated, except that Federated is explicitly removed during the compilation. Additionally, the spider




  [Read more...]
Database Sharding at Netlog, with MySQL and PHP
+0 Vote Up -0Vote Down

This article accompanies the slides from a presentation on database sharding. Sharding is a technique used for horizontal scaling of databases we are using at Netlog. If you’re interested in high performance, scalability, MySQL, php, caching, partitioning, Sphinx, federation or Netlog, read on …

This presentation was given at the second day of FOSDEM 2009 in Brussels. FOSDEM is an annual conference on open source software with about 5000 hackers. I was invited by Kris Buytaert and Lenz Grimmer to give a talk in the MySQL Dev

  [Read more...]
Database sharding at Netlog (FOSDEM talk slides)
+0 Vote Up -0Vote Down

Here are the slides from yesterday’s presentation about horizontal database scaling through sharding at the mySQL dev room at FOSDEM 2009.

I’ve got a ton of notes and remarks to these slides, which will become available here soon.

FOSDEM 2009 - Boosting performance with MySQL 5.1 partitions
Employee +0 Vote Up -0Vote Down

The MySQL Developers Room at FOSDEM was a success. There was a full room (about 80 seats, and many sitting on the floor) throughout the day. We started at 9am with a talk on PBXT, and ended at 17:00. My talk had a large audience and was very well received.

The enthusiasm was quite high also because of a stunt I did during the presentation. It was a creative way of explaining the difference between physical and logical partitioning. I will repeat this stunt at the MySQL Users Conference, during my Tutorial on Partitioning.
Those who were

  [Read more...]
FOSDEM 2009 - Boosting performance with MySQL 5.1 partitions
Employee +0 Vote Up -0Vote Down

The MySQL Developers Room at FOSDEM was a success. There was a full room (about 80 seats, and many sitting on the floor) throughout the day. We started at 9am with a talk on PBXT, and ended at 17:00. My talk had a large audience and was very well received.

The enthusiasm was quite high also because of a stunt I did during the presentation. It was a creative way of explaining the difference between physical and logical partitioning. I will repeat this stunt at the MySQL Users Conference, during my Tutorial on Partitioning.
Those who

  [Read more...]
FOSDEM 2009 - Boosting performance with MySQL 5.1 partitions
Employee +0 Vote Up -0Vote Down

The MySQL Developers Room at FOSDEM was a success. There was a full room (about 80 seats, and many sitting on the floor) throughout the day. We started at 9am with a talk on PBXT, and ended at 17:00. My talk had a large audience and was very well received.

The enthusiasm was quite high also because of a stunt I did during the presentation. It was a creative way of explaining the difference between physical and logical partitioning. I will repeat this stunt at the MySQL Users Conference, during my Tutorial on Partitioning.
Those who

  [Read more...]
MySQL Webinar on Partitioning — by Use Case Competition Winner
Employee +0 Vote Up -0Vote Down

Do you remember Guy Adams? He was one of the winners of the “5.1 Use Case Competition”, ending up on position #2. Guess what: He has a webinar coming up tomorrow, by the title Deploying MySQL in a High Performance Satellite Network Management Environment by Parallel.
 
Guy works with Parallel Ltd. in Milton Keynes in the UK. You may also want to read up on

  [Read more...]
Some partition movement tricks with MySQL 5.1
+0 Vote Up -0Vote Down

I have been using partitioning in MySQL 5.1 since it has been in late beta, and with the odd bump I’ve been quite satisfied with its stability and ease of use. Perhaps the days of maintaining nasty application logic to maintain tables like table_yyyymm manually are finally over!

While some control freaks out there might still prefer the painful approach of manually keeping dated tables and using merge tables/views to link them together, you still have a lot of flexibility in how you manage individual partitions with the ALTER TABLE REORGANIZE/CHANGE/ADD PARTITION commands.

One trick i’ve discovered to work pretty well, is that you can swap in and out partitions right out from under MySQL’s nose. This can be useful in some situations.

WARNING!! This is not supported by MySQL to the best of my knowledge, and possibly is a risky, bad idea. Proceed only if you think you

  [Read more...]
Q&A on MySQL 5.1
+0 Vote Up -0Vote Down
Listening to Sheeri's presentation on MySQL 5.1, I saw that there are a few questions left unanswered. I am listing here some of the questions that I found interesting, plus a few from an early webinar on the same topic.

Q: does Partitioning physically split data?A: No. Some engines (MyISAM, Archive) do a physical split, but this is not necessary, as you see if you apply partitioning to a InnoDB table. Partitioning is a logical split of data, for easy retrieval. It is completely transparent to the user.Q: Can you set partitions to different servers?A: No. Partitions are logical parts of one table within one server. Partitioning through the Federated engine is not supported.Q: How efficient are Row-Based Replication operations compared to Statement based

  [Read more...]
New in MySQL 5.1: Sheeri’s Presentation
+0 Vote Up -0Vote Down

In a nutshell: What’s New in MySQL 5.1.

Release notes: Changes in release 5.1.x (Production).

And yes, very early on (at about two minutes in), I talk about my take on Monty’s controversial post at Oops, we did it again.

To play the video directly, go to http://technocation.org/node/663/play. To download the 146 Mb video to your computer for offline playback, go to http://technocation.org/node/663/download. The slides can be downloaded as a

  [Read more...]
Tutorial on Partitioning at the MySQL Users Conference 2009
+0 Vote Up -0Vote Down



Another approval has arrived. I will be co-presenting a tutorial at the MySQL Users Conference 2009.
Partitioning in MySQL 5.1. My co-speaker is Sarah Sproehnle, a well known MySQL trainer. The topic is one of the flagship features of MySQL 5.1.

The tutorial will cover the basics, several practical scenarios, tips and tricks, and some sneak peeks of the latest enhancements being developed.
Backing up MySQL partitioned tables is nothing special
+0 Vote Up -0Vote Down
In case you were wondering, after seeing this self-defined tutorial, backing up MySQL 5.1 partitioned tables does not require any particular additional technology.
This tutorial presents MySQL 5.1 in a short paragraph, and then starts talking about a commercial backup application, as if it were the only way of backing up partitioned tables.
The title, for sure, is misleading. Backing Up MySQL Partitioned Tables. Reading such a title, readers expect either a list of possible solutions, or being presented with the only existing way of dealing with this task. Instead, the only instructions that you get is how to use the above mentioned tool, which is definitely not the only way of getting the job done. It is also doubly misleading because, since partitioning is a new technology,

  [Read more...]
Recovery features for ALTER TABLE of partitioned tables
Employee +0 Vote Up -0Vote Down
A feature which hasn't been so public about the implementation
of partitioning is the support for atomicity of many ALTER TABLE
statements using partitioned tables.

This atomicity exists for
ALTER TABLE ADD PARTITION ....
ALTER TABLE REORGANIZE PARTITION ...
ALTER TABLE DROP PARTITION ...
ALTER TABLE COALESCE PARTITION

Given that partitioning often works with very large tables it
was desirable to have a higher level of security for ALTER TABLE
of partitioned tables. To support this a DDL log was implemented.
This DDL log will in future versions be used also for many other
meta data statements. The DDL log will record all files added,
renamed and dropped during an ALTER TABLE command as above.

The design is done in such a way that the ALTER TABLE will either
fail and then all

















  [Read more...]
Previous 30 Newer Entries Showing entries 31 to 60 of 90 Next 30 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.