Planet MySQL Planet MySQL: Meta Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 10 of 15 5 Older Entries

Displaying posts with tag: storage engine api (reset)

An old note on the Storage Engine API
+1 Vote Up -0Vote Down

Whenever I stick my head into the MySQL storage engine API, I’m reminded of a MySQL User Conference from several years ago now.

Specifically, I’m reminded of a slide from an early talk at the MySQL User Conference by Paul McCullagh describing developing PBXT. For “How to write a Storage Engine for MySQL”, it went something like this:

  • Develop basic INSERT (write_row) support – INSERT INTO t1 VALUES (42)
  • Develop full table scan (rnd_init, rnd_next, rnd_end)  - SELECT * from t1
  • If you’re sane, stop here.
  • A lot of people stop at step 3. It’s a really good place to stop too. It avoids most of the tricky parts that are unexpected, undocumented and unlogical (yes, I’m inventing words here).

    MySQL vs Drizzle plugin APIs
    +1 Vote Up -0Vote Down

    There’s a big difference in how plugins are treated in MySQL and how they are treated in Drizzle. The MySQL way has been to create a C API in front of the C++-like (I call it C- as it manages to take the worst of both worlds) internal “API”. The Drizzle way is to have plugins be first class citizens and use exactly the same API as if they were inside the server.

    This means that MySQL attempts to maintain API stability. This isn’t something worth trying for. Any plugin that isn’t trivial quickly surpasses what is exposed via the C API and has to work around it, or, it’s a storage engine and instead you have this horrible mash of C and C++. The byproduct of this is that no core server features are being re-implemented as plugins. This means

      [Read more...]
    The EXAMPLE storage engine
    +2 Vote Up -0Vote Down

    The Example storage engine is meant to serve mainly as a code example of the stub of a storage engine for example purposes only (or so the code comment at the start of ha_example.cc reads). In reality however, it’s not very useful. It likely was back in 2004 when it could be used as a starting point for starting some simple new engines (my guess would be that more than a few of the simpler engines started from ha_example.cc).

    The sad reality is the complexity of the non-obviousness of the bits o the storage engine API you actually care about are documented in ha_ndbcluster.cc, ha_myisam.cc and ha_innodb.cc. If you’re doing something that isn’t already done by one of those three engines: good luck.

    Whenever I looked at ha_example.cc I always wished there was something

      [Read more...]
    The MEMORY storage engine
    +3 Vote Up -0Vote Down

    I recently wrote about Where are they now: MySQL Storage Engines and The MERGE storage engine: not dead, just resting…. or forgotten. Today, it’s the turn of the MEMORY storage engine – otherwise known as HEAP.

    This is yet another piece of the MySQL server that sits largely unmaintained and unloved. The MySQL Manual even claims that it supports encryption… with the caveat of having to use the SQL functions for encryption/decryption rather than in the engine itself (so,

      [Read more...]
    Where are they now: MySQL Storage Engines
    +3 Vote Up -0Vote Down

    There was once a big hooplah about the MySQL Storage Engine Architecture and how it was easy to just slot in some other method of storage instead of the provided ones. Over the years I’ve repeatedly mentioned how this

      [Read more...]
    Is your Storage Engine buggy or the database server?
    +5 Vote Up -0Vote Down

    If your storage engine returns an error from rnd_init (or doStartTableScan as it’s named in Drizzle) and does not save this error and return it in any subsequent calls to rnd_next, your engine is buggy. Namely it is buggy in that a) an error may not be reported back to the user and b) everything may explode horribly when rnd_next is called after rnd_init returned an error.

    Unless it is running on MariaDB 5.2 or (soon, when the patch hits the tree) Drizzle.

    Monty (Widenius, not Taylor) wrote a patch for MariaDB based on my bug report that

      [Read more...]
    A more complete look at Storage Engine API
    +3 Vote Up -0Vote Down

    Okay… So I’ve blogged many times before about the Storage Engine API in Drizzle. This API is somewhat inherited from MySQL. We have very much attempted to make it a much cleaner interface. Our goals in making changes include: make it much easier to write and maintain a storage engine, make the upper layer code obviously correct and clear in what it’s doing and being able to more easily introduce optimisations.

    I’ve recently added a Storage Engine that is only used in testing: storage_engine_api_tester. I’ve blogged on it producing call graphs (really state transition graphs) before both for Storage Engine and Cursor.

    I’ve been expanding the test. My

      [Read more...]
    Cursor states
    +2 Vote Up -0Vote Down

    Following on from my post yesterday on the various states of a Storage Engine, I said I’d have a go with the Cursor object too. A Cursor is used by the Drizzle kernel to get and set data in a table. There can be more than one cursor open at once, and more than one per thread. If your engine cannot cope with this, it is its responsibility to figure it out and return the appropriate errors.

    Let’s look at a really simple operation, inserting a couple of rows and then reading them back via a full table scan.

    Now, this graph is slightly incomplete as there is no doEndTableScan() call. But you can see

      [Read more...]
    Storage Engine API state graph
    +2 Vote Up -0Vote Down

    Drizzle still has a number of quirks inherited from the MySQL Storage Engine API (e.g. BLOBs, row buffer, CREATE SELECT and lack of DDL transaction boundaries, key tuple format). One of the things we fixed a long time ago was to have proper methods for StorageEngines to be called for: startTransaction, startStatement, endStatement, commit and rollback.

    If you’ve had to implement a transactional storage engine in MySQL you will be well aware of the pattern

      [Read more...]
    BLOBS in the Drizzle/MySQL Storage Engine API
    +3 Vote Up -0Vote Down

    Another (AFAIK) undocumented part of the Storage Engine API:

    We all know what a normal row looks like in Drizzle/MySQL row format (a NULL bitmap and then column data):

    Nothing that special. It’s a fixed sized buffer, Field objects reference into it, you read out of it and write the values into your engine. However, when you get to BLOBs, we can’t use a fixed sized buffer as BLOBs may be quite large. So, the format with BLOBS is the bit in the row is a length of the blob (1, 2, 3 or 4 bytes – in Drizzle it’s only 3 or 4 bytes now and soon only 4 bytes once we fix a bug that isn’t interesting to discuss here). The Second part of the

      [Read more...]
    Showing entries 1 to 10 of 15 5 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.