Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Previous 30 Newer Entries Showing entries 31 to 60 of 65 Next 5 Older Entries

Displaying posts with tag: PBXT (reset)

The PrimeBase BLOB Streaming (PBMS) engine alpha version 5.08 is ready
+0 Vote Up -0Vote Down
Alpha version 5.08 of the BLOB streaming engine for MySQL has been released. You can download the source code from www.blobstreaming.org/download. The documentation has also been updated.

What's new in 5.08:
  • All PBMS data is stored under a 'pbms' directory in the MySQL server's data directory rather than in the database directories them selves.
  • This version now builds with Drizzle and can be loaded as a 'Blobcontainer' plug-in.
  • Added the possibility of storing BLOB metadata along with the BLOB in the repository.
  • Added the possibility of assigning an alias to a BLOB, which can then be used to retrieve the BLOB instead of using the engine generated URL.
  • Added an updateable system table 'pbms_metadata_header' to control which HTTP headers are stored as

  [Read more...]
Solving the PBXT DBT2 Scaling Problem
+0 Vote Up -0Vote Down
One little bit of wisdom I would like to pass on:

If a program runs fast with 20 threads, that does not mean it will run fast with 50. And if it runs fast with 50, it does not mean that it will run fast with 100, and if it runs fast with 100 ... don't bet on it running fast with 200 :)

In my last blog I discussed some improvement to the performance of PBXT running the DBT2 benchmark. Despite the overall significant increase in performance I noted a drop off at 32 threads that indicated a scaling problem. For the last couple of weeks I have been working on this problem and I have managed to fix it:

As before, this test was done

  [Read more...]
Understanding the various MySQL Products & Variants
+0 Vote Up -0Vote Down

The MySQL marketplace today is far more complex then simply choosing between a particular version of MySQL that Sun/MySQL (http://mysql.com) produces.
The MySQL server product in general is released under the GNU General Public License (GPL) v2, however you should carefully review the MySQL Legal Policies (http://www.mysql.com/about/legal/) as a number of exceptions and different license agreements operate for companion tools such as MySQL Cluster, MySQL client libraries and documentation for example.

Looking into the MySQL ecosystem for products, I’ve produced the following categories:

  • Sun/MySQL Official Products
    • MySQL Versions
  • MySQL Variants
    • Community
    • Enterprise
  • MySQL Plugins

  [Read more...]
Improving PBXT DBT2 Performance
+0 Vote Up -0Vote Down
DBT2, with over 40% conflicts, is an very challenging benchmark, especially for an MVCC based engine. And, as a result, it is not a test that an engine is automatically good at. InnoDB has been extensively optimized for DBT2, and it shows.

For the last few weeks I have had the opportunity to focus on PBXT DBT2 performance for the first time. I started with a memory bound DBT2 test and the current state of this work is illustrated below.

These results were achieved using MySQL 5.1.30 on an 8 core, 64-bit, Linux machine with an SSD drive and a 5 warehouse DBT2 database.

The dip off at 32 threads is left as an exercise for the reader :) Patches will be excepted!

So what were the major changes that lead to

  [Read more...]
PBMS supports Drizzle
+1 Vote Up -0Vote Down
The PBMS engine now works with Drizzle. Well actually it has been working with Drizzle for several months since I have been using Drizzle as my 'host' server while adding new features to the engine. I will tell you about the new features in future posts.
Hooks for a 'blobcontainer' type plug-in have been added to Drizzle that allow a plug-in to catch insert. update, or delete operations on BLOB columns in any table and handle the BLOB storage itself. The plug-in gets called above the storage engine level so it is independent of the storage engines. This is very similar to the way that PBMS works in MySQL 5.1 with engines other than PBXT where it uses triggers to perform the same function. But the way it is done with Drizzle is a lot more efficient.
So in Drizzle the PBMS engine is both a 'blobcontainer' plug-in as well as a storage engine. 
Surprisingly little needed to be

  [Read more...]
MySQL Magazine - Winter 2009 Released!!
+0 Vote Up -0Vote Down

This is one of the largest issues so far.  In addition, it has some of the best content we have ever had. The articles are all in-depth with some exciting new information:

  • Introduction to XtraDB: an overview of the new XtraDB storage engine along with benchmarks and information about planned future improvements
  • Changes in the MySQL DBA and Developer Exams for Version 5.1: what’s coming in the new exams covering MySQL Server 5.1
  • Covering Indexes in MySQL: how to create indexes that optimize query execution
  • PBXT’s Coder’s Guide: going in-depth on how you can work with the code for the new PBXT storage engine
  • Coding Corner: Peter’s regular column continues his look at transaction time validity

It is

  [Read more...]
PBXT goes RC!
+0 Vote Up -0Vote Down
With all the booha about MySQL not being ready for GA, it makes me almost afraid to announce, ahem, ... and PBXT is, ehr, RC.

It has been just over a year now since I started developing the fully durable version of PBXT. Before that, PBXT was Beta. After that, it was Alpha again.

Now we have 2 solid Beta versions behind us, Vladimir and I have fixed all known bugs for this version, including quite a number of foreign key bugs. We have all 259 mysql-test-run tests that were adapted for PBXT (and a bunch of our own) running through without any errors on 4 platforms: Mac OS X, Linux 32-bit and 64-bit, and Windows. Our buildbot is giving us a green light, at last!

Besides this we have done crash tests, load tests and crash and load tests (I mean recovery)! And maybe most important, we have it ticking away in a very

  [Read more...]
xtstat: Tells you exactly what PBXT is doing!
+0 Vote Up -0Vote Down
I have created a new tool, called xtstat, for analyzing the performance of the PBXT storage engine.

The way it works is simple. PBXT now counts all kinds of things: transactions committed and rolled back, statements executed, records read and written, tables and indexes scanned, bytes read, written and flushed to various types of files: record, index, data logs, transaction logs, and so on.

A SELECT on the system table PBXT.STATISTICS (or INFORMATION_SCHEMA.PBXT_STATISTICS if PBXT was built inside the MySQL tree) returns the current totals of all these counters. xtstat does a SELECT every second on this table and prints the difference. In this way, you can see how much work PBXT is doing in each area.

There are currently 48 different statistics:

To ensure all

  [Read more...]
PBXT 1.0.06 Beta Released
+0 Vote Up -0Vote Down
On friday we released the second Beta version of PBXT. PBXT is a transactional storage engine for MySQL 5.1 and 6.0. You can find out more about the engine at www.primebase.org.

PBXT is pluggable, so it can be built separately from the MySQL tree, and loaded dynamically at runtime using the LOAD PLUGIN statement.

You can download PBXT from here. A "quick guide" to building and installing the plugin is provided. I have also updated the documentation for this version.

There are no major new features in this release because we are working towards the RC version in December. But we wrote some release notes to prove we have been busy :)

There is now also a version of PBXT available for

  [Read more...]
Percona’s patches spread to a wider audience
+0 Vote Up -0Vote Down

Percona's patches are now available to a wider audience via OurDelta, a community effort to provide  builds with features (Percona patches, Google patches, etc) and storage engines (PBXT, Sphinx, etc) that aren't in the main MySQL server. Arjen Lentz is really the brainchild behind this. Kudos Arjen!

What does this mean for the Percona patches? Well, now you can get them in more places.  But it doesn't change our own commitment to keep innovating in ways our customers (and we ourselves) find useful.  We're still building our own builds and hosting the

  [Read more...]
PBXT moves to Launchpad
+0 Vote Up -0Vote Down
It's been a week or 2 and some of you may already have heard that PBXT has moved from Sourceforge to Lauchpad.net: https://launchpad.net/pbxt.

There are several very good reasons for the move, not the least of which is that MySQL has already moved to Launchpad, and Drizzle is there too. It simply makes sense for a storage engine like PBXT to be on the same platform.

And check this out, Stewart Smith has already ported PBXT to Drizzle. You will find the tree here: PBXT in Drizzle. I will be pulling Stewart's changes back into the PBXT

  [Read more...]
PBXT Beta Version Released!
+0 Vote Up -0Vote Down
I am pleased to announce that the Beta version of PBXT has just been released. You can download the source code of the storage engine from www.primebase.org/download. I have also updated the documentation for this version.

Configuring and building the engine is easier than ever now. To configure PBXT all you have to do is specify the path to the MySQL source code tree (after building MySQL), for example:

./configure --with-mysql=/home/foo/mysql/mysql-5.1.26-rc

The PBXT configure command will retrieve all required options from the MySQL build. For example whether to do a debug or optimized build and where to install the plugin are determined automatically, depending on how you configured MySQL.

This was a source

  [Read more...]
Mutex contention and other bottlenecks in MySQL
+0 Vote Up -0Vote Down
Over the last few weeks I have been doing some work on improving the concurrency performance of PBXT. The last Alpha version (1.0.03) has quite a few problems in this area.

Most of the problems have been with r/w lock and mutex contention but, I soon discovered that MySQL has some serious problems of it's own. In fact, I had to remove some of the bottlenecks in MySQL in order to continue the optimization of PBXT.

The result for simple SELECT performance is shown in the graph below.

Here you can see that the gain is over 60% for 32 or more concurrent threads. Both results show the performance with the newly optimized version of PBXT. The test is running on a 2.16 MHz dual core

  [Read more...]
PBXT compiles without change under MySQL 5.1.25!
+0 Vote Up -0Vote Down
OK, now I know that the GA version of 5.1 is rapidly approaching. PBXT compiles with the latest release of MySQL without any changes!

This has never been the case before. Just search the PBXT code for MYSQL_VERSION_ID, and you will find things like:
and, even worse:
#if MYSQL_VERSION_ID >= 60005
The lack of changes that affect pluggable storage engines can only mean that the bug fixes required are diminishing in scope.

And I believe this is a far better gauge of whether GA is close than any other

  [Read more...]
MySQL Conference and Expo 2008, Day Three
+0 Vote Up -0Vote Down

Here’s a rundown of Thursday (day 3) of the MySQL Conference and Expo. This day’s sessions were much more interesting to me than Wednesday’s, and in fact I wanted to go to several of them in a single time slot a couple of times.

Inside the PBXT Storage Engine

This session was, as it sounds, a look at the internals of PBXT, a transactional storage engine for MySQL that has some interesting design techniques. I had been looking forward to this session for a while, and Paul McCullagh’s nice explanations with clear diagrams were a welcome aid to understanding how PBXT works. Unlike some of the other storage engines, PBXT is being developed in full daylight, with an emphasis on community involvement and input. (Indeed, I may be contributing to it myself, in order to make its monitoring

  [Read more...]
MySQL Conference Liveblogging: Introduction To The BLOB Streaming Project (Wednesday 3:00PM)
+0 Vote Up -0Vote Down
  • Paul McCullagh presents
  • BLOB
    • invented by Jim Starkey
    • Basic Large OBject
    • Binary Large OBject
    • photos, films, mp4 files, pdfs, etc
  • how MySQL handles BLOBs
    • mysql client send buffer -> receive buffer on the server (max_allowed_packet)
    • streaming a BLOB
      • continuous data stream
      • stream BLOB data directly in and out of the database
      • store BLOBs of any size (>4GB) in the database
      • create a scalable back-end that can handle any throughput and storage requirements. Wouldn't need to know in advance how big the database will get
      • provide an open system that can be used by all engines
      • provide extensions for BLOB streaming to existing MySQL clients
  • why put BLOBs
  [Read more...]
MySQL Conference and Expo 2008, Day One
+0 Vote Up -0Vote Down

Today is the first day at the conference (aside from the tutorials, which were yesterday). Here’s what I went to:

New Subquery Optimizations in 6.0

By Sergey Petrunia. This was a similar session to one I went to last year. MySQL has a few cases where subqueries are badly optimized, and this session went into the details of how this is being addressed in MySQL 6.0. There are several new optimization techniques for all types of subqueries, such as inside-out subqueries, materialization, and converting to joins. The optimizations apply to scalar subqueries and subqueries in the FROM clause. Performance results are very good, depending on which data you choose to illustrate. The overall point is that the worst-case subquery nastiness should be resolved. I’m speaking of WHERE NOT IN(SELECT…) and friends. It remains to be seen how

  [Read more...]
The pursuit of a synchronous world
+0 Vote Up -0Vote Down

Well at least your MySQL database world.

As Paul eluded to, PrimeBase Technologies has a project to provide synchronous replication for MySQL in a High Availability environment. It is more then an idea, there is a plan.

Is it possible?
What are the use cases?
How can you use it?
Would you use it?

Some input to date. We need these questions and more, and we seeking more input for discussion.

Unfortunately the opportunity to hear any input during a presentation is left to the last day of the conference, so we have created a BoF session on Tuesday night for a round table discussion if necessary. We encourage people to bring specific cases and situations for feedback, the reasons why MySQL Replication, MySQL Cluster, DRBD/HeatBeat or any other solution does not satisfy your needs, and

  [Read more...]
Replication is dead, long live Replication!
+0 Vote Up -0Vote Down
Brian Aker has found general agreement with his post: "The Death of Read Replication".

Arjen Lentz says "I think Brian is right...", and Frank Mash confirmed: "what Brian says about replication, caching and memcached is very true".

Just like Video killed the Radio Star it looks like maybe Memcached killed the Replication Hierarchy!

But of course, Brian and others are talking about replication for scaling reads.

In my session on PBXT next week at the conference I will be talking about how we plan to use

  [Read more...]
Storage Engines at the MySQL Conference
+0 Vote Up -0Vote Down

I’ll be following closely the progression of Storage Engines available in the MySQL Database server, well soon to be available when 5.1 gets to GA (hopefully by end of Q2 which is what we have been told). Tick, Tick, time is running out.

PrimeBase XT (PBXT) and Blob Streaming is obviously my clear focus, actually now working for PrimeBase Technologies, the company which I want to note for people is an Open Source company, committed at providing an open source alternative to the other commercial players. You also have at the MySQL Conference talks on the the existing InnoDB from Innobase (a subsidiary of market RDBMS leader Oracle). There is a

  [Read more...]
Come and get a t-shirt at UC2008
+0 Vote Up -0Vote Down

Here I am at my desk sporting the PrimeBase supporters t-shirt that will be available at the exhibitors booth at the 2008 MySQL Conference. The front is rather uneventful with the official logo, but the back will be worth the experience. So everybody interested in supporting PBXT as the transactional storage engine for MySQL developed by the community and for the community, please come and see us and mention the secret password.

We have been placed way back in the right hand side of the exhibitors hall at booth 518, in front of the Open Source and OEM providers.

PrimeBase Technologies - Day 1
+0 Vote Up -0Vote Down

Today I started my new job at PrimeBase Technologies. The company that has brought you the PBXT and Blob Streaming Pluggable Storage Engines for MySQL 5.1.

My move to Germany has gone mostly without incident and now I’m settling in to different weather, language and food, plus the change in time zones +6 hours.

A smaller company from my previous, but I’m part of a larger group then expected. One of 26 people in the office. It’s good to have a desk, a big monitor (and definitely not a German keyboard) and see and talk to people on various topics and interests in comparison to either past work at home by myself, or on a new customer site each week during my consulting days.

Preparations for the upcoming MySQL

  [Read more...]
A day with Paul McCullagh - Architect of PBXT and BlobStreaming
+0 Vote Up -0Vote Down

Continuing on from my lightning visits with Jan Kneschke and Michael Zinner, today I got to spend a day with Paul McCullagh at his home in Hamburg Germany.

Paul is the architect of the PBXT Pluggable Storage Engine for MySQL 5.1, and also the Blob Streaming Storage Engine. His work was acknowledged with the MySQL Community contributor for the year in 2007. The successful PrimeBase product for the publishing Industry in Europe and North America also now uses PBXT for underlying data storage which is

  [Read more...]
Just what are MySQL 9.x features?
+0 Vote Up -0Vote Down

Top marks to Jay Pipes for getting the Forge 2.0 finally out after quite some time, as well as in the midst of the MySQL Conference he is organizing.

I am worried however about some of the content, as shown in the screenshot below, the opening page lists Worklog tasks/features for versions 6.x or 7.x , that’s ok, but features in 9.x. Where is the practicality of thinking more then 2 releases ahead, and just having a future bucket. Indeed, we have 5.1 and 6.0 already frozen and not releases, so 6.x is already 3 releases out.

Tonight we were told at the NY PHP Meeting MySQL 5.1 is not due to late Q2, so that’s at least June 2008.
The MySQL 5.1 Release

  [Read more...]
My passion for Open Source
+0 Vote Up -0Vote Down

I am a very strong proponent of Open Source (excluding my Macbook). Joining MySQL Inc was a wonderful achievement, being part of the team behind the most popular open source database. Leaving MySQL was not an easy decision due to the people, but the Sun transition and requirements did help. However it is no surprise I am joining another open source company - Primebase Technologies in Hamburg, Germany. My association with the MySQL Community will only be strengthened with my full work and support behind the PBXT and Blob Streaming pluggable storage engines for MySQL.

It is actually poetic that I am joing Primebase for I have the auspicious recognition while an active part of the MySQL community of introducing Paul McCullagh to MySQL some 2 years ago.

  [Read more...]
New version and a new home for PBXT!
+0 Vote Up -0Vote Down
I have just released the first fully durable version of PBXT. Because of the amount of new code I have reverted PBXT to Alpha status. This version, 1.0-alpha, can be downloaded from: http://www.primebase.org/download.

Oh, which reminds me: PBXT now has a new home at http://www.primebase.org, so take a look around! I have actually found a bit of time to write some documentation. Right now the documentation describes building, installation, and the PBXT system parameters. Future additions will include information on performance tuning and a road map for PBXT development.

But there is more to the new home than just a new web-site. The PBXT project is now owned and funded by PrimeBase Technologies, an open source software

  [Read more...]
PBXT & DBT2: Dubugging C/C++ 101
+0 Vote Up -0Vote Down
Yesterday I starting testing PBXT using the DBT2 benchmark. Following the implementation of durability and SELECT FOR UPDATE for the engine I was more interested in the benchmark as a test for stability and concurrency than performance. I was not disappointed...

Which bug first?

Well I immediately ran into 3 bugs. Isn't it funny how bugs often come in batches, which leaves you thinking: "Oh sh.. where do I start?". Here's my advice: start with the bug that is most likely to disappear if you fix the others!

A simple example, you have 2 bugs: an unexpected exception is occurring, and you're loosing memory. First look for the memory loss, because it may disappear when you fix the exception (because you may be loosing memory in the error handler).

Take things one problem at time:

Another thing: once you have decided for one of the bugs,

  [Read more...]
Making PBXT Fully Durable
+0 Vote Up -0Vote Down
Until now PBXT has been ACId (with a lower-case d). This is soon to change as I have had some weeks to work on a fully durable version of the transactional engine (http://www.primebase.com/xt).

My first concern in making PBXT fully durable was to what extent I would have to abandon the original "write-once" design. While there are a number of ways to implement durability, the only method used by databases (as far as I know) is the write-ahead log.

The obvious advantage of this method is that all changes can be flushed at once. However, this requires that all data be written twice: once to the log and after that, to the database itself.

My solution to this problem is a compromise, but I think it is a good one. In a nutshell: short records are written twice, and long records are written once. When it comes to

  [Read more...]
New PBXT/MyBS release enables JDBC-based BLOB streaming!
+0 Vote Up -0Vote Down
This is quite a milestone for me! At last it possible to actually do some practical work with the BLOB streaming engine (MyBS)!

For this release I have completed changes to the MySQL Connector/J 5.0.7, to allow BLOB data to be transparently stored and retrieved from the MyBS BLOB repository. The new version of the driver is called MySQL Connector/J SE (streaming enabled).

Uploading a BLOB is as simple as using setBinaryStream() or setBlob() on INSERT or UPDATE. By using getBinaryStream() or getBlob() after a SELECT you get direct access to the data stream coming from the repository. More information and some examples are provided in the documentation at: http://www.blobstreaming.org/documentation.

To try this out you need to install the latest versions of PBXT and MyBS. Both are

  [Read more...]
BLOB streaming engine (MyBS), version 0.5 Alpha released!
+0 Vote Up -0Vote Down
With some effort just before my holiday, I have managed to complete the release of the next version of MyBS, the BLOB streaming engine for MySQL.

This version includes all the basic functionality required to stream BLOB data in and out of MySQL tables.

The main features are:
  • Uploading of BLOB data directly into the database using HTTP PUT or GET methods.

  • Downloaded of BLOB data directly from the database using HTTP GET.

  • BLOB size may exceed 4GB - theoretical BLOB size limit of 256 Terabytes.

  • BLOBs are stored in a repository which manages references from other storage engine tables.

  • BLOBs are referenced by a URL.

  • URLs referencing BLOBs in the repository have a unique access code, for security.

  • The theoretical maximum repository size is 4 Zettabytes (2^72

  [Read more...]
Previous 30 Newer Entries Showing entries 31 to 60 of 65 Next 5 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.