Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 171 Next 30 Older Entries

Displaying posts with tag: code (reset)

and now for something completely different…
+0 Vote Up -0Vote Down

As many of you know, I’ve been working in the MySQL (http://www.mysql.com) world for quite a while now. IN fact, it was nearly 10 years ago when I first started hacking on MySQL Cluster (https://www.mysql.com/products/cluster/) at MySQL AB.

Most recently, I was at Percona which was a wonderful journey where over my nearly three years there the company at least doubled in size, launched several new software products and greatly improved the quality and frequency of releases.

However the time has come for something

  [Read more...]
Converting MySQL trees to git
+0 Vote Up -0Vote Down

I have put up a set of scripts on github: https://github.com/stewartsmith/bzr-to-git-conversion-scripts. Why do I need these? Well… if only bzr fast-export|git fast-import worked flawlessly for large, complex and old trees. It doesn’t.

Basically, when you clone this repo you can run “./sync-BLAH.sh” and it’ll pull BZR trees for the project, convert to git and clean things up a bit. You will likely have to edit the sync-BLAH.sh scripts as I have them pointed at branches on my own machine (to speed up the process, not having to do fresh BZR branches of MySQL trees over the network is a feature - it’s never been fast.). You’ll also want to edit the

  [Read more...]
The road to Percona Server 5.6
+3 Vote Up -0Vote Down

Over a year ago now, I announced the first Percona Server 5.6 alpha on the Percona MySQL Performance Blog (Announcing Percona Server 5.6 Alpha). That was way back on August 14th, 2012 and it was based on MySQL 5.6.5 released in April.

I’m really happy now to point to the release of the first GA release of Percona Server 5.6 along with

  [Read more...]
The end of Bazaar
+2 Vote Up -0Vote Down

I’ve used the Bazaar (bzr) version control system since roughly 2005. The focus on usability was fantastic and the team at Canonical managed to get the entire MySQL BitKeeper history into Bazaar – facilitating the switch from BitKeeper to Bazaar.

There were some things that weren’t so great. Early on when we were looking at Bazaar for MySQL it was certainly not the fastest thing when it came to dealing with a repository as large as MySQL. Doing an initial branch over the internet was painful and a much worse experience than BitKeeper was. The work-around that we all ended up using was downloading a tarball of a recent Bazaar repository and then “bzr pull” to get the latest. This was much quicker than

  [Read more...]
Disk usage: bzr vs git
+2 Vote Up -0Vote Down

For MySQL 5.1, 5.5 and 5.6 in the same repository, after repacking:

bzr: 269MB (217MB pack, 52MB indicies)

git: 177MB repo (152MB pack)

One thing I’ll say is that BZR is always more chatty over the network and is substantially slower than GIT in pulling a fresh copy.

Who is working on MySQL 5.7?
+5 Vote Up -0Vote Down

First I find out the first commit that is in 5.7 that isn’t in 5.6 (using bzr missing) and then look at the authors of all of those commits. Measuring the number of commits is a really poor metric as it does not measure the complexity of the code committed, and if your workflow is to revise a patchset before committing, you get much fewer commits than if you commit 10 times a day.

There are a good number of people who are committing a lot of code to the latest MySQL development tree. (Sorry for the annoying layout of “count. number-of-commits name”)

  • 1022 Magnus Blaudd
  • 723 Jonas Oreland
  • 329 Marko Mäkelä
  • 286 Krunal Bauskar
  • 230 Tor Didriksen
  • 218 John David Duncan
  • 205 Vasil Dimov
  • 197
  •   [Read more...]
    Who is working on MariaDB 10.0?
    +0 Vote Up -0Vote Down

    There was some suggestion after my previous post (Who works on MariaDB and MySQL?) that I look at MariaDB 10.0 – so I have. My working was very simple, in a current MariaDB 10.0 BZR tree (somewhat beyond 10.0.3), I ran the following command:

    bzr log -n0 -rtag:mariadb-10.0.0..|egrep '(author|committer): '| \
      sed -e 's/^\s*//; s/committer: //; s/author: //'| \
      sort -u|grep -iv oracle

     

    MariaDB foundation/MontyProgram/SkySQL:

  • Alexander Barkov
  • Alexey Botchkov
  • Daniel Bartholomew
  • Elena Stepanova
  • Igor Babaev
  • Jani Tolonen
  • knielsen
  • Michael Widenius
  • sanja
  • Sergei Golubchik
  •   [Read more...]
    nanomysql – tiny MySQL client lib
    +0 Vote Up -0Vote Down

    I recently got pointed towards https://github.com/shodanium/nanomysql/ which is a tiny (less than 400 lines of C++) MySQL client library which is GPL licensed.

    If you need to link into non-GPL compatible code, there is the (slightly larger and full featured) libdrizzle library. But if you want something *tiny* and are okay with GPL, then nanomysql may be something to look at.

    Who works on MariaDB and MySQL?
    +1 Vote Up -0Vote Down

    Looking at the committers/authors of patches in the bzr tree for MariaDB 5.5.31.

    Non Oracle Contributors:

  • Alexander Barkov
  • Alexey Botchkov
  • Elena Stepanova
  • Igor Babaev
  • knielsen
  • Michael Widenius
  • sanja
  • Sergei Golubchik
  • Sergey Petrunya
  • timour
  • Vladislav Vaintroub
  • Oracle (as they pull Oracle changes):

  • Aditya A
  • Akhila Maddukuri
  • Alexander Nozdrin
  • Anirudh Mangipudi
  • Annamalai Gurusami
  • Astha Pareek
  • Balasubramanian Kandasamy
  • Chaithra Gopalareddy
  • Daniel Fischer
  • Gleb Shchepa
  • Harin Vadodaria
  • Hery Ramilison
  • Igor Solodovnikov
  • Inaam Rana
  • Jon Olav Hauglid
  •   [Read more...]
    An old note on the Storage Engine API
    +1 Vote Up -0Vote Down

    Whenever I stick my head into the MySQL storage engine API, I’m reminded of a MySQL User Conference from several years ago now.

    Specifically, I’m reminded of a slide from an early talk at the MySQL User Conference by Paul McCullagh describing developing PBXT. For “How to write a Storage Engine for MySQL”, it went something like this:

  • Develop basic INSERT (write_row) support – INSERT INTO t1 VALUES (42)
  • Develop full table scan (rnd_init, rnd_next, rnd_end)  - SELECT * from t1
  • If you’re sane, stop here.
  • A lot of people stop at step 3. It’s a really good place to stop too. It avoids most of the tricky parts that are unexpected, undocumented and unlogical (yes, I’m inventing words here).

    MySQL vs Drizzle plugin APIs
    +1 Vote Up -0Vote Down

    There’s a big difference in how plugins are treated in MySQL and how they are treated in Drizzle. The MySQL way has been to create a C API in front of the C++-like (I call it C- as it manages to take the worst of both worlds) internal “API”. The Drizzle way is to have plugins be first class citizens and use exactly the same API as if they were inside the server.

    This means that MySQL attempts to maintain API stability. This isn’t something worth trying for. Any plugin that isn’t trivial quickly surpasses what is exposed via the C API and has to work around it, or, it’s a storage engine and instead you have this horrible mash of C and C++. The byproduct of this is that no core server features are being re-implemented as plugins. This means

      [Read more...]
    Some storage engine features you only get if you’re InnoDB
    +1 Vote Up -0Vote Down

    I had reason to look into the extended secondary index code in MariaDB and MySQL recently, and there was one bit that I really didn’t like.

    MariaDB:

    share->set_use_ext_keys_flag(legacy_db_type == DB_TYPE_INNODB);

    MySQL:

    use_extended_sk= (legacy_db_type == DB_TYPE_INNODB);

    In case you were wondering what “legacy_db_type” actually does, let me tell you: it’s not legacy at all, it’s kind of key to how the whole “metadata” system in MySQL works. For example, to drop a table, this magic number is used to work out what storage engine to call to drop the table.

    Now, these code snippets

      [Read more...]
    The EXAMPLE storage engine
    +2 Vote Up -0Vote Down

    The Example storage engine is meant to serve mainly as a code example of the stub of a storage engine for example purposes only (or so the code comment at the start of ha_example.cc reads). In reality however, it’s not very useful. It likely was back in 2004 when it could be used as a starting point for starting some simple new engines (my guess would be that more than a few of the simpler engines started from ha_example.cc).

    The sad reality is the complexity of the non-obviousness of the bits o the storage engine API you actually care about are documented in ha_ndbcluster.cc, ha_myisam.cc and ha_innodb.cc. If you’re doing something that isn’t already done by one of those three engines: good luck.

    Whenever I looked at ha_example.cc I always wished there was something

      [Read more...]
    The ARCHIVE Storage Engine
    +5 Vote Up -0Vote Down

    I wonder how much longer the ARCHIVE storage engine is going to ship with MySQL…. I think I’m the last person to actually fix a bug in it, and that was, well, a good number of years ago now. It was created to solve a simple problem: write once read hardly ever. Useful for logs and the like. A zlib stream of rows in a file.

    You can actually easily beat ARCHIVE for INSERT speed with a non-indexed MyISAM table, and with things like TokuDB around you can probably get pretty close to compression while at the same time having these things known as “indexes”.

    ARCHIVE for a long time held this niche though and was widely and quietly used (and likely still is). It has the great benefit of being fairly lightweight – it’s only about 2500 lines of code (1130 if

      [Read more...]
    The MySQL Cluster storage engine
    +2 Vote Up -0Vote Down

    This is one close to my heart. I’ve recently written on other storage engines: Where are they now: MySQL Storage EnginesThe MERGE storage engine: not dead, just resting…. or forgotten and The MEMORY storage engine. Today, it’s the turn of MySQL Cluster.

    Like InnoDB, MySQL Cluster started outside of MySQL. Those of you paying attention at home may notice a correlation between storage engines not written exclusively for MySQL and being at all successful.

    NDB

      [Read more...]
    TokuDB
    +1 Vote Up -0Vote Down

    Big news at Percona Live MySQL Conference and Expo this week, Tokutek open sourced TokuDB thus making my previous post Where are they now: MySQL Storage Engines out of date in just a few days.

    In this case, I really don’t mind. It’s rather exciting that they’ve gone ahead and done this –  and it’s not just a code drop: https://github.com/Tokutek/ft-engine is where things are at, and recent commits were 2hrs and 18hrs ago which means that this is being maintained. This is certainly a good way to grow a developer community.

    While being a MySQL engine is really interesting, the MongoDB

      [Read more...]
    A few notes on InnoDB in MySQL 5.7.1
    +7 Vote Up -0Vote Down

    I’ve started poking around the MySQL 5.7.1 source tree (although just from tarball as I don’t see a BZR tree yet). I thought I’d share a few thoughts:

    • InnoDB temporary tables. Not REDO logged. What does this mean? It’s a huge step in removing the dependency on MEMORY and MyISAM engines for temporary tables used in query execution. With InnoDB temporary tables there is no reason for MEMORY engine to continue to exist, there is absolutely no way in which it is better.
    • InnoDB temp tables aren’t insert buffered
      This probably doesn’t really matter as you’re not going to be doing REDO logging for them (plus things are generally short lived)… but it could be a future area for performance improvement
    • The

      [Read more...]
    Where are they now: MySQL Storage Engines
    +3 Vote Up -0Vote Down

    There was once a big hooplah about the MySQL Storage Engine Architecture and how it was easy to just slot in some other method of storage instead of the provided ones. Over the years I’ve repeatedly mentioned how this

      [Read more...]
    30 configuration options and counting
    +1 Vote Up -2Vote Down

    While Domas may have rather effictively trolled the discussion with his post on howto configure table/user statistics (which gave me a good chuckle I do have to say), it’s at least incorrect for Percona Server as you have to enable the “userstat” server option :)

    That being said, once enabled there are no extra configuration variables to think about. This is a huge advantage over configuring PERFORMANCE_SCHEMA - which has a total of THIRTY configuration options (31 if you include the global enable/disable option).

    Some of these thirty odd configuration variables are only going to matter if

      [Read more...]
    Refactoring Internal temporary tables (another stab at it)
    +1 Vote Up -2Vote Down

    A few weekends ago, I started to again look at the code in Drizzle for producing internal temporary tables. Basically, we have a few type of tables:

    • Standard
    • Temporary (from CREATE TEMPORARY TABLE)
    • Temporary (from ALTER TABLE)
    • Internal temporary (to help with query execution)

    If you’re lucky enough to be creating one of the first three types, you go through an increasingly lovely pile of code that constructs a nice protobuf message about what the table should look like and hands all responsibility over to the storage engine as to how to do that. The basic idea is that Drizzle gets the heck out of the way and lets the storage engine do its thing. This code path looks rather different

      [Read more...]
    diffstat of MySQL 5.6 versus 5.5
    +0 Vote Up -0Vote Down

    Yesterday I wrote about what the diffstat between MySQL 5.5 and MariaDB 5.5 was, and previously to that, about the MariaDB code size as reported by sloccount. Let’s look at MySQL 5.6.

    A naive wc based “lines of code” for MySQL 5.6 sql/ directory is ~490kLOC which contasts with MySQL 5.5 being ~375kLOC by the same measure. If we diffstat the sql/ directory like I did for MariaDB 5.5 we get:

     357 files changed, 172871 insertions(+), 67922 deletions(-)

    Versus, as you remember from yesterday for MariaDB 5.5 over MySQL 5.5:

     250 files changed, 83639
      [Read more...]
    So what about the diffstat of MariaDB compared to MySQL?
    +0 Vote Up -4Vote Down

    So, I’ve looked at what sloccount says on the differences between Oracle MySQL over versions of itself and the various MySQL branches around. What I haven’t looked at is the diffstat. Firstly, let’s look at MariaDB.

    I’m going to look at MariaDB 5.5.29 as compared to MySQL 5.5.29, both checked out from bzr. A naive diffstat would give us:

     5261 files changed, 1086165 insertions(+), 122751 deletions(-)

    And this looks like an awful lot of code that has changed: about 1,086,165 lines! This actually includes a whole other copy of InnoDB in the form of XtraDB. If we take that into account we get:

     5032 files changed, 864997 insertions(+), 125099 deletions(-)

    Which is still incredibly

      [Read more...]
    Which is bigger: MySQL or PostgreSQL?
    +1 Vote Up -0Vote Down

    From my previous posts, we have some numbers (excluding NDB) for the size of MySQL, so what about PostgreSQL? Here, I used PostgreSQL git trunk and classing things in the contrib/ directory as plugins. I put the number of lines of code in the src/backend/storage directory down as storage engines LoC but did not count it as non-kernel code.

    Version Total LoC Plugin LoC Storage Engines LoC Remaining (kernel) MySQL 5.5.30 858,441 2,706 171,009 684,726 (79% kernel) MySQL 5.6.10 1,049,344 29,122 236,067 784,155 (74% kernel) MariaDB 5.5 1,142,118 11,781 304,015 826,322 (72% kernel) Drizzle trunk 334,810 31,150 130,727 172,933 (51% kernel) PostgreSQL trunk 648,691 61,934 17,802 586,757  [Read more...]
    Is MySQL bigger than Linux?
    +0 Vote Up -0Vote Down

    I’m going to take the numbers from my previous post, MySQL Modularity, Are We There Yet? for the “kernel” size of MySQL – that is, everything that isn’t a plugin or storage engine.

    For Linux kernel, I’m just going to use the a-bit-old git tree I have on my laptop. I’ve decided that the following directories are for “plugins” drivers/ arch/ sound/ firmware/ crypto/ usr/ virt/ tools/ scripts/ fs/*/* and everything else is core kernel code.

    Version Total LoC Total Plugin LoC Remaining (kernel) MySQL 5.6.10 1,049,344 265,189 784,155 (74% kernel) MariaDB 5.5 1,142,118  [Read more...]
    MySQL modularity, are we there yet?
    +2 Vote Up -1Vote Down

    MySQL is now over four times the size than it was with MySQL 3.23. This has not come in the shape of plugins.

    Have we improved modularity over time? I decided to take LoC count for plugins and storage engines (in the case of Drizzle, memory, myisam and innobase are storage engines and everything else comes under plugin). I’ve excluded NDB from these numbers as it is rather massive and is pretty much still a separate thing.

    Version Total LoC Plugin LoC Storage Engines LoC Remaining (kernel) MySQL 3.23.58 371,987 0 (0%) 176,276 195,711 (52% kernel) MySQL 5.1.68 721,331 228 237,124 483,979 (67% kernel) MySQL 5.5.30 858,441 2,706 171,009 684,726 (79% kernel)  [Read more...]
    Other MySQL branch code sizes
    +1 Vote Up -0Vote Down

    Continuing on from my previous posts, MySQL code size over releases and MariaDB code size I’ve decided to also look into some other code branches. I’ve used the same methodology as my previous few posts: sloccount for C and C++ code only.

    There are also other branches around in pretty widespread use (if only within a single company). I grabbed the Google, Facebook and Twitter patches and examined them too, along with Percona Server 5.1 and 5.5.

    Codebase LoC (C, C++) +/- from MySQL Google v4 patch 5.0.37 970,110 +26,378 (from MySQL 5.0.37) MySQL@Facebook 1,087,715 +15,768  [Read more...]
    MySQL code size over releases
    +7 Vote Up -1Vote Down

    As the start of a bit of a delve into the various MySQL branches and patch sets that have been around, let’s start looking at the history of MySQL itself. This is how big MySQL has been over all of the major releases since the beginning (where beginning=3.23). (edit: These numbers were all gathered using sloccount and only counting C++ and C source files.)

    Codebase LoC (C, C++) +/- from previous MySQL MySQL 3.23.58 371,987 0 MySQL 4.0.30 368,695 -3,292 (from MySQL 3.23) MySQL 4.1.24 859,572 +490,877 (from MySQL 4.0)
    +174,352 excluding NDB MySQL 5.0.96 916,667 +57,095 (from MySQL 4.1) MySQL 5.1.68 1,052,636 +135,969 (from MySQL 5.0) MySQL 5.5.30 1,189,747 +137,111 (from MySQL 5.1) MySQL 5.6.10 1,544,202 +354,455 (from
      [Read more...]
    Fun with Coverity found bugs (Episode 1)
    +2 Vote Up -0Vote Down

    Taking the inspiration of Valeriy Kravchuk great series of blog posts “Fun with Bugs” (and not http://funwithbugs.com/ which is about both caring for and eating bugs), and since I recently went and run Coverity against Drizzle, I thought I’d have a small series of posts on bugs that it has found (and I’ve fixed).

    An idea that has been pervasive in the Drizzle project (and one that I rather subscribe to) is that there is two types of correct: correct and obviously correct. Being obviously correct is much,

      [Read more...]
    Coverity scan for Drizzle
    +2 Vote Up -2Vote Down

    Coverity is a static analysis tool, which although proprietary itself does offer a free scanning service for free and open source software (which is great by the way, I totally owe people who make that happen a frosty beverage).

    Prompted by someone volunteering to get MariaDB into the Coverity Scan, I realized that I hadn’t actually followed through with this for Drizzle. So, I went and submitted Drizzle. As a quick overview, this is the number of problems of each severity both projects got back:

    Severity MariaDB Drizzle High 178 96 Medium 1020 495 Low 47 52

    I don’t know what MySQL may be, but it’d be great to see this out in the open too.

    On the clarity of licensing
    +2 Vote Up -0Vote Down

    Being clear in licensing is a great benefit to users. I’m quite proud that with Drizzle we got to a really simple situation:

    • server is GPLv2
    • client libraries are BSD licensed and are fresh implementations

    We had the complete okay from Sun to have libdrizzle be BSD licensed and for us to work on that.

    Showing entries 1 to 30 of 171 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.