Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 683 Next 30 Older Entries

Displaying posts with tag: drizzle (reset)

The joy of Unicode
+1 Vote Up -0Vote Down

So, back in late 2008, rather soon after we got to start working on Drizzle full time, someone discovered unicodesnowmanforyou.com, or:

Since we had decided that Drizzle was going to be UTF-8 everywhere,(after seeing for years how hard it was for people to get character sets correct in MySQL) we soon added ☃.test to the tree, which tried a few interesting things:

CREATE TABLE ☃; CREATE DATABASE ☃; etc etc

Because what better to show off UTF-8 than using odd Unicode characters for table names, database names and file names. Well… it turns out we were all good except if you attempted to check out the source tree on Solaris. It was some combination of Python, Bazaar and Solaris that meant you


  [Read more...]
Is Drizzle dead?
+0 Vote Up -0Vote Down
Yesterday someone opened a Launchpad question asking "is Drizzle dead?".  I have answered that question on Launchpad but wanted to blog about it to give a bit of background context.

As I am sure most of the people who read this know, Drizzle is an Open Source database which was originally forked from the alpha version of MySQL 6.0.  At the time it was an extremely radical approach to Open Source development, many features were stripped out and re-written as plugins to turn it into a micro-kernel style architecture.  Every merge request was automatically throughly tested on several platforms for regressions, memory leaks and even positive/negative performance changes.

In fact Drizzle has influenced many Open Source projects



  [Read more...]
Ghosts of MySQL Past: Part 2
+0 Vote Up -0Vote Down

This continues on from my post yesterday and also contains content from my linux.conf.au 2014 talk (view video here).

Way back in May in the year 2000, a feature was added to MySQL that would keep many people employed for many years – replication. In 3.23.15 you could replicate from one MySQL instance to another. This is commonly cited as the results of two weeks of work by one developer. The idea is simple: create a log of all the SQL queries that modify the database and then replay them on a slave. Remember, this is

  [Read more...]
Converting MySQL trees to git
+0 Vote Up -0Vote Down

I have put up a set of scripts on github: https://github.com/stewartsmith/bzr-to-git-conversion-scripts. Why do I need these? Well… if only bzr fast-export|git fast-import worked flawlessly for large, complex and old trees. It doesn’t.

Basically, when you clone this repo you can run “./sync-BLAH.sh” and it’ll pull BZR trees for the project, convert to git and clean things up a bit. You will likely have to edit the sync-BLAH.sh scripts as I have them pointed at branches on my own machine (to speed up the process, not having to do fresh BZR branches of MySQL trees over the network is a feature - it’s never been fast.). You’ll also want to edit the

  [Read more...]
The end of Bazaar
+2 Vote Up -0Vote Down

I’ve used the Bazaar (bzr) version control system since roughly 2005. The focus on usability was fantastic and the team at Canonical managed to get the entire MySQL BitKeeper history into Bazaar – facilitating the switch from BitKeeper to Bazaar.

There were some things that weren’t so great. Early on when we were looking at Bazaar for MySQL it was certainly not the fastest thing when it came to dealing with a repository as large as MySQL. Doing an initial branch over the internet was painful and a much worse experience than BitKeeper was. The work-around that we all ended up using was downloading a tarball of a recent Bazaar repository and then “bzr pull” to get the latest. This was much quicker than

  [Read more...]
Who is working on MariaDB 10.0?
+0 Vote Up -0Vote Down

There was some suggestion after my previous post (Who works on MariaDB and MySQL?) that I look at MariaDB 10.0 – so I have. My working was very simple, in a current MariaDB 10.0 BZR tree (somewhat beyond 10.0.3), I ran the following command:

bzr log -n0 -rtag:mariadb-10.0.0..|egrep '(author|committer): '| \
  sed -e 's/^\s*//; s/committer: //; s/author: //'| \
  sort -u|grep -iv oracle

 

MariaDB foundation/MontyProgram/SkySQL:

  • Alexander Barkov
  • Alexey Botchkov
  • Daniel Bartholomew
  • Elena Stepanova
  • Igor Babaev
  • Jani Tolonen
  • knielsen
  • Michael Widenius
  • sanja
  • Sergei Golubchik
  •   [Read more...]
    nanomysql – tiny MySQL client lib
    +0 Vote Up -0Vote Down

    I recently got pointed towards https://github.com/shodanium/nanomysql/ which is a tiny (less than 400 lines of C++) MySQL client library which is GPL licensed.

    If you need to link into non-GPL compatible code, there is the (slightly larger and full featured) libdrizzle library. But if you want something *tiny* and are okay with GPL, then nanomysql may be something to look at.

    MySQL vs Drizzle plugin APIs
    +1 Vote Up -0Vote Down

    There’s a big difference in how plugins are treated in MySQL and how they are treated in Drizzle. The MySQL way has been to create a C API in front of the C++-like (I call it C- as it manages to take the worst of both worlds) internal “API”. The Drizzle way is to have plugins be first class citizens and use exactly the same API as if they were inside the server.

    This means that MySQL attempts to maintain API stability. This isn’t something worth trying for. Any plugin that isn’t trivial quickly surpasses what is exposed via the C API and has to work around it, or, it’s a storage engine and instead you have this horrible mash of C and C++. The byproduct of this is that no core server features are being re-implemented as plugins. This means

      [Read more...]
    Libdrizzle 5.1.4 released!
    +0 Vote Up -0Vote Down
    I've just created a release of Libdrizzle 5.1.4 (the BSD licensed C connector for MySQL servers) which for me is one of the most exciting releases to date.

    Why?
    Not because the amount of features or anything that we have added, in fact this is mostly a minor release with bug fixes.  It is because Brian Aker and I didn't submit the most code for it.  That honour goes to Wim Lewis from The Omni Group who has done a fantastic job fixing up Libdrizzle.

    As for the release itself the main fixes revolve around cleaning up code and many fixes to the server-side prepared statement handling.  Behind the scenes Wim has supplied many improvements to the test suite and Brian has setup




      [Read more...]
    The MEMORY storage engine
    +3 Vote Up -0Vote Down

    I recently wrote about Where are they now: MySQL Storage Engines and The MERGE storage engine: not dead, just resting…. or forgotten. Today, it’s the turn of the MEMORY storage engine – otherwise known as HEAP.

    This is yet another piece of the MySQL server that sits largely unmaintained and unloved. The MySQL Manual even claims that it supports encryption… with the caveat of having to use the SQL functions for encryption/decryption rather than in the engine itself (so,

      [Read more...]
    The MERGE storage engine: not dead, just resting…. or forgotten.
    +0 Vote Up -0Vote Down

    Following on from my fun post on Where are they now: MySQL Storage Engines, I thought I’d cover the few storage engines that are really just interfaces to a collection of things. In this post, I’m talking about MERGE.

    The MERGE engine was basically a multiplexer down to a number of MyISAM tables. They all had to be the same, there was no parallel query execution and it saw fairly limited use. One of the main benefits was that then you could actually put more rows in a MyISAM table than your “files up to 2/4GB” file system allowed. With the advent of partitioning, this really should have instantly gone away

      [Read more...]
    Where are they now: MySQL Storage Engines
    +3 Vote Up -0Vote Down

    There was once a big hooplah about the MySQL Storage Engine Architecture and how it was easy to just slot in some other method of storage instead of the provided ones. Over the years I’ve repeatedly mentioned how this

      [Read more...]
    30 configuration options and counting
    +1 Vote Up -2Vote Down

    While Domas may have rather effictively trolled the discussion with his post on howto configure table/user statistics (which gave me a good chuckle I do have to say), it’s at least incorrect for Percona Server as you have to enable the “userstat” server option :)

    That being said, once enabled there are no extra configuration variables to think about. This is a huge advantage over configuring PERFORMANCE_SCHEMA - which has a total of THIRTY configuration options (31 if you include the global enable/disable option).

    Some of these thirty odd configuration variables are only going to matter if

      [Read more...]
    Refactoring Internal temporary tables (another stab at it)
    +1 Vote Up -2Vote Down

    A few weekends ago, I started to again look at the code in Drizzle for producing internal temporary tables. Basically, we have a few type of tables:

    • Standard
    • Temporary (from CREATE TEMPORARY TABLE)
    • Temporary (from ALTER TABLE)
    • Internal temporary (to help with query execution)

    If you’re lucky enough to be creating one of the first three types, you go through an increasingly lovely pile of code that constructs a nice protobuf message about what the table should look like and hands all responsibility over to the storage engine as to how to do that. The basic idea is that Drizzle gets the heck out of the way and lets the storage engine do its thing. This code path looks rather different

      [Read more...]
    MySQL and the forks in the road
    +0 Vote Up -0Vote Down
    There's a lot of activity in the MySQL ecosystem currently. I've written an article for The H which takes a look at MySQL and the various forks and spinoffs to give an overview of where MySQL's community and close competition is currently at.
    http://www.h-online.com/open/features/MySQL-and-the-forks-in-the-road-1829242.html
    Which is bigger: MySQL or PostgreSQL?
    +1 Vote Up -0Vote Down

    From my previous posts, we have some numbers (excluding NDB) for the size of MySQL, so what about PostgreSQL? Here, I used PostgreSQL git trunk and classing things in the contrib/ directory as plugins. I put the number of lines of code in the src/backend/storage directory down as storage engines LoC but did not count it as non-kernel code.

    Version Total LoC Plugin LoC Storage Engines LoC Remaining (kernel) MySQL 5.5.30 858,441 2,706 171,009 684,726 (79% kernel) MySQL 5.6.10 1,049,344 29,122 236,067 784,155 (74% kernel) MariaDB 5.5 1,142,118 11,781 304,015 826,322 (72% kernel) Drizzle trunk 334,810 31,150 130,727 172,933 (51% kernel) PostgreSQL trunk 648,691 61,934 17,802 586,757  [Read more...]
    MySQL modularity, are we there yet?
    +2 Vote Up -1Vote Down

    MySQL is now over four times the size than it was with MySQL 3.23. This has not come in the shape of plugins.

    Have we improved modularity over time? I decided to take LoC count for plugins and storage engines (in the case of Drizzle, memory, myisam and innobase are storage engines and everything else comes under plugin). I’ve excluded NDB from these numbers as it is rather massive and is pretty much still a separate thing.

    Version Total LoC Plugin LoC Storage Engines LoC Remaining (kernel) MySQL 3.23.58 371,987 0 (0%) 176,276 195,711 (52% kernel) MySQL 5.1.68 721,331 228 237,124 483,979 (67% kernel) MySQL 5.5.30 858,441 2,706 171,009 684,726 (79% kernel)  [Read more...]
    Other MySQL branch code sizes
    +1 Vote Up -0Vote Down

    Continuing on from my previous posts, MySQL code size over releases and MariaDB code size I’ve decided to also look into some other code branches. I’ve used the same methodology as my previous few posts: sloccount for C and C++ code only.

    There are also other branches around in pretty widespread use (if only within a single company). I grabbed the Google, Facebook and Twitter patches and examined them too, along with Percona Server 5.1 and 5.5.

    Codebase LoC (C, C++) +/- from MySQL Google v4 patch 5.0.37 970,110 +26,378 (from MySQL 5.0.37) MySQL@Facebook 1,087,715 +15,768  [Read more...]
    unireg.h is finally gone
    +2 Vote Up -0Vote Down

    I got rid of unireg.cc way back in 2009 as I rewrote all the FRM related code inside Drizzle to instead use a nice protobuf based structure. If you’re wondering what was there, I just quote this part of pack_screens() from unireg.cc in MySQL 5.6:

    start_row=4; end_row=22; cols=80; fields_on_screen=end_row+1-start_row;

    We have gradually pulled things out of unireg.h over the years too. But, let’s go back to ask the question “What is UNIREG?”. To answer that, I’m going to quote from something that was current back when MySQL 3.22 was the latest and greatest:

    In 1979, he developed an in-house database tool called UNIREG for managing databases. Since 1979, UNIREG has

      [Read more...]
    First version of Drizzle Tools for MySQL servers released
    +1 Vote Up -0Vote Down
    Today marks the first release of Drizzle Tools for MySQL servers.  Drizzle Tools aims to be a collection of useful utilities to use with MySQL servers based around the work on the Libdrizzle Redux project.

    In this first version there is one utility in the tree called 'drizzle-binlogs'.  If you've seen me talk about this tool before it is because it used to be included in the Libdrizzle 5.1 source but has now been moved here to be developed independently.  For those who haven't 'drizzle-binlogs' is a tool which connects to a MySQL server as a slave, retrieves the binary log files and stores them locally.  This could be used as part of a backup solution or a rapid way to help create a new MySQL master server.

    Due to the API changes before the Libdrizzle API became stable



      [Read more...]
    Fun with Coverity found bugs (Episode 1)
    +2 Vote Up -0Vote Down

    Taking the inspiration of Valeriy Kravchuk great series of blog posts “Fun with Bugs” (and not http://funwithbugs.com/ which is about both caring for and eating bugs), and since I recently went and run Coverity against Drizzle, I thought I’d have a small series of posts on bugs that it has found (and I’ve fixed).

    An idea that has been pervasive in the Drizzle project (and one that I rather subscribe to) is that there is two types of correct: correct and obviously correct. Being obviously correct is much,

      [Read more...]
    Coverity scan for Drizzle
    +2 Vote Up -2Vote Down

    Coverity is a static analysis tool, which although proprietary itself does offer a free scanning service for free and open source software (which is great by the way, I totally owe people who make that happen a frosty beverage).

    Prompted by someone volunteering to get MariaDB into the Coverity Scan, I realized that I hadn’t actually followed through with this for Drizzle. So, I went and submitted Drizzle. As a quick overview, this is the number of problems of each severity both projects got back:

    Severity MariaDB Drizzle High 178 96 Medium 1020 495 Low 47 52

    I don’t know what MySQL may be, but it’d be great to see this out in the open too.

    Developing Libdrizzle
    +2 Vote Up -2Vote Down
    This weekend I am supposed to be giving a talk at FOSDEM on Libdrizzle.  Unfortunately my kids and I all fell ill on Thursday (my wife appears to be immune) so I had to cancel my plans (infecting 5000 people didn't seem wise :)

    Instead I am writing this blog post about Libdrizzle and my part in it which covers some of what I was going to talk about.

    History of Libdrizzle

    Libdrizzle started out as a from-scratch C connector for Drizzle and MySQL originally created by Eric Day.  It was designed to be high performance and use common standards to make it easy to work on.  In the summer of 2010 it was merged into the main Drizzle tree where development has been focused.  There were several attempts to split it out again but until now none were truly successful.

    For a few





      [Read more...]
    Introducing Drizzle Tools
    +1 Vote Up -2Vote Down
    As part of the Libdrizzle Redux project I created an example tool which was bundled with it which will connect to a MySQL server as a slave and download the binary logs to local files.  This was developed as a quick example of what can be done with the new binlog API.

    Two things quickly became apparent:
  • We shouldn't really be distributing applications in a library
  • I am going to be developing more useful tools around libdrizzle and they certainly shouldn't be in the same package
  • BSD is a fantastic license for a library, but I personally prefer GPLv2 for applications
  • With this in mind I have pulled the drizzle_binlogs utility from Libdrizzle trunk (and therefore won't be in the 5.1.3 release) and put it in its own repository.  It has been licensed appropriately (GPLv2) and I am already beginning to develop more tools to go with it.


      [Read more...]
    Libdrizzle 5.1.2 released and Libdrizzle at FOSDEM
    +1 Vote Up -0Vote Down
    Libdrizzle is getting better and better with every release, and to reflect this we have two announcements to make.

    First of all, Libdrizzle 5.1.2 has been released.  This release has several major bug fixes and performance enhancements.  Changes are as follows:
    • Non-blocking Windows connections are now more stable
    • Improvements to Windows building
    • Unix Socket connections are now more stable
    • Memory allocation/freeing cleanups (for performance and ease of code)
    • Network packet buffer now much more flexible
    • Many performance improvements (the bundled drizzle_binlogs tool is now around 10x faster on my i7 laptop)
    • Several other minor bug fixes
    API chages:
    • drizzle_query_str() has been removed, drizzle_query() with a 0 byte length parameter now does the same thing.
    As



      [Read more...]
    On the clarity of licensing
    +2 Vote Up -0Vote Down

    Being clear in licensing is a great benefit to users. I’m quite proud that with Drizzle we got to a really simple situation:

    • server is GPLv2
    • client libraries are BSD licensed and are fresh implementations

    We had the complete okay from Sun to have libdrizzle be BSD licensed and for us to work on that.

    Questions about MariaDB JDBC Driver
    +6 Vote Up -0Vote Down
    The recent release of the MariaDB client libraries has prompted questions about their purpose as well as provenance.  Colin Charles posted that some of these would be answered in the very near future.  I have a couple of specific questions about the MariaDB JDBC driver, which I hope will be addressed at that time.  
    1.) What is really in the MariaDB JDBC driver and how exactly does it differ from the drizzle JDBC driver?  What, if any, relation is there to Connector/J code?  There is a 
      [Read more...]
    The state of MySQL client libraries
    +5 Vote Up -0Vote Down

    Those who’ve been around the MySQL world are probably aware of the much-discussed topics of GPL licensing, dual licensing, and in particular, licensing of the client libraries (also called connectors or drivers) and the FOSS exception (http://www.mysql.com/about/legal/licensing/foss-exception/) to that licensing. This is newly relevant with the announcement of a permissively-licensed MySQL-compatible client library for MariaDB.

    The difference is that this time there’s been some question about the provenance and history of the source code. Some people asked me about this. Some of them were aware of a relatively obscure detail: there’ve been permissively licensed MySQL client libraries for years, in the form of libdrizzle, a BSD-licensed library for the Drizzle fork of MySQL.

    Here are some of the thoughts that seemed to be going through

      [Read more...]
    A MySQL Christmas present - Libdrizzle 5.1.0
    +2 Vote Up -0Vote Down
    Brian Aker and I have been working hard in the last few weeks to give you a great Christmas present, Libdrizzle 5.1.0.  The MySQL compatible, BSD licensed C connector (so static compiling with commercial software gets the thumbs up!).

    The latest changes include:
    • A server-side prepared statement API
    • Improved binary log API
    • An example binary log remote retrieval utility using the binlog API called "drizzle_binlogs"
    • A new build system, DDM4 which is used by Gearman and Memcached
    • Many bugs fixes
    The source and manuals can be found on the Launchpad downloads page.  Please enjoy, feel free to file bugs, questions and hack on code on our Launchpad page.  Happy holidays to all!
    Libdrizzle Redux 5.0-alpha1 Released!
    +1 Vote Up -0Vote Down
    Over the past few months I have been spending my spare time on a new project.  A new version of libdrizzle which is much simpler to use and with many new features.  Today the first version of this MySQL compatible client is released, called Libdrizzle Redux.

    Why 5.0?  Because Libdrizzle 1.0 and 2.0 have already been released, in packaging versions 3.0 and 4.0 used as API revisions.  So 5.0 is the fresh start.


    Main Features

    These are the main features of the library:

    • A BSD licensed MySQL compatible C connector, so you can statically link it with commercial software
    • A simplified API compared to Libdrizzle.  No more confusion over whether the client or library should be allocating/freeing.  There isn't a big difference to the MySQL C API for most things.
    • New






      [Read more...]
    Showing entries 1 to 30 of 683 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.