One little bit of wisdom I would like to pass on:
If a program runs fast with 20 threads, that does not mean it
will run fast with 50. And if it runs fast with 50, it does not
mean that it will run fast with 100, and if it runs fast with 100
... don't bet on it running fast with 200 :)
In my last blog I discussed some improvement to the
performance of PBXT running the DBT2 benchmark. Despite the
overall significant increase in performance I noted a drop off at
32 threads that indicated a scaling problem. For the last couple
of weeks I have been working on this problem and I have managed
to fix it:
As before, this test was done using MySQL
5.1.30 on an 8 core, 64-bit, Linux machine with an SSD drive and
a 5 warehouse DBT2 …
The MySQL marketplace today is far more complex then simply
choosing between a particular version of MySQL that Sun/MySQL produces.
The MySQL server product in general is released under the
GNU General Public License (GPL) v2, however
you should carefully review the MySQL Legal
Policies as a number of exceptions and different license
agreements operate for companion tools such as MySQL Cluster,
MySQL client libraries and documentation for example.
Looking into the MySQL ecosystem for products, I’ve produced the following categories:
- Sun/MySQL Official Products
- MySQL Versions
- MySQL Variants
- Community
- Enterprise
DBT2, with over 40% conflicts, is an very challenging benchmark,
especially for an MVCC based engine. And, as a result, it is not
a test that an engine is automatically good at. InnoDB has been
extensively optimized for DBT2, and it shows.
For the last few weeks I have had the opportunity to focus on
PBXT DBT2 performance for the first time. I started with a memory
bound DBT2 test and the current state of this work is illustrated
below.
These results were achieved using MySQL 5.1.30 on
an 8 core, 64-bit, Linux machine with an SSD drive and a 5
warehouse DBT2 database.
The dip off at 32 threads is left as an exercise for the reader
:) Patches will be excepted!
So what were the major changes that lead to this
improvement?
Don't Wait Too Long!
When I began the optimizations, PBXT …
The PBMS engine now works with Drizzle. Well actually it has been
working with Drizzle for several months since I have been using
Drizzle as my 'host' server while adding new features to the
engine. I will tell you about the new features in future
posts.
Hooks for a 'blobcontainer' type plug-in have been added to
Drizzle that allow a plug-in to catch insert. update, or delete
operations on BLOB columns in any table and handle the BLOB
storage itself. The plug-in gets called above the storage engine
level so it is independent of the storage engines. This is very
similar to the way that PBMS works in MySQL 5.1 with engines
other than PBXT where it uses triggers to perform the same
function. But the way it is done with Drizzle is a lot more
efficient.
So in Drizzle the PBMS engine is both a 'blobcontainer' plug-in
as well as a storage engine.
Surprisingly little needed to be done to the PBMS engine to get
it to build with …
This is one of the largest issues so far. In addition, it has some of the best content we have ever had. The articles are all in-depth with some exciting new information:
- Introduction to XtraDB: an overview of the new XtraDB storage engine along with benchmarks and information about planned future improvements
- Changes in the MySQL DBA and Developer Exams for Version 5.1: what’s coming in the new exams covering MySQL Server 5.1
- Covering Indexes in MySQL: how to create indexes that optimize query execution
- PBXT’s Coder’s Guide: going in-depth on how you can work with the code for the new PBXT storage engine
- Coding Corner: Peter’s regular column continues his look at transaction time validity
It is available for download, along with all …
[Read more]
With all the booha about MySQL not being ready for GA, it makes
me almost afraid to announce, ahem, ... and PBXT is,
ehr, RC.
It has been just over a year now since I started developing the
fully durable version of PBXT. Before that, PBXT was Beta. After
that, it was Alpha again.
Now we have 2 solid Beta versions behind us, Vladimir and I have
fixed all known bugs for this version, including quite a number
of foreign key bugs. We have all 259 mysql-test-run tests that
were adapted for PBXT (and a bunch of our own) running through
without any errors on 4 platforms: Mac OS X, Linux 32-bit and
64-bit, and Windows. Our buildbot is giving us a green light, at
last!
Besides this we have done crash tests, load tests and crash and
load tests (I mean recovery)! And maybe most important, we have
it ticking away in a very demanding OEM product …
I have created a new tool, called xtstat, for analyzing the
performance of the PBXT storage engine.
The way it works is simple. PBXT now counts all kinds of things:
transactions committed and rolled back, statements executed,
records read and written, tables and indexes scanned, bytes read,
written and flushed to various types of files: record, index,
data logs, transaction logs, and so on.
A SELECT on the system table PBXT.STATISTICS (or
INFORMATION_SCHEMA.PBXT_STATISTICS if PBXT was built inside the
MySQL tree) returns the current totals of all these counters.
xtstat does a SELECT every second on this table and prints the
difference. In this way, you can see how much work PBXT is doing
in each area.
There are currently 48 different statistics:
To ensure all this counting does not itself cost any performance,
each thread counts for itself, so no locking is required. The
SELECT on …
On friday we released the second Beta version of PBXT. PBXT is a
transactional storage engine for MySQL 5.1 and 6.0. You can find
out more about the engine at www.primebase.org.
PBXT is pluggable, so it can be built separately from the MySQL
tree, and loaded dynamically at runtime using the LOAD PLUGIN
statement.
You can download PBXT from here. A
"quick guide" to building and installing the plugin is provided.
I have also updated the documentation for this version.
There are no major new features in this release because we are
working towards the RC version in December. But we wrote some
release notes to prove we have been busy :)
There is now also a version of PBXT available for …
Percona's patches are now available to a wider audience via OurDelta, a community effort to provide builds with features (Percona patches, Google patches, etc) and storage engines (PBXT, Sphinx, etc) that aren't in the main MySQL server. Arjen Lentz is really the brainchild behind this. Kudos Arjen!
What does this mean for the Percona patches? Well, now you can get them in more places. But it doesn't change our own commitment to keep innovating in ways our customers (and we ourselves) find useful. We're still building our own builds and …
[Read more]
It's been a week or 2 and some of you may already have heard that
PBXT has
moved from Sourceforge to Lauchpad.net: https://launchpad.net/pbxt.
There are several very good reasons for the move, not the least
of which is that MySQL has already moved to Launchpad, and
Drizzle is there too. It simply makes sense for a
storage engine like PBXT to be on the same platform.
And check this out, Stewart Smith has already ported PBXT to
Drizzle. You will find the tree here: PBXT in …