Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 245 Next 30 Older Entries

Displaying posts with tag: Programming (reset)

GUUG Frühjahrsfachgespräch 2014: CfP ends on May 31st!
Employee +0 Vote Up -0Vote Down

The German Unix User Group (GUUG) will hold their annual conference "Frühjahrsfachgespräch" on September 23-26 this year (I know, not really "Frühjahr" anymore, but this is how it is).

The Call for Presentations is still open until May 31st. Talks can be proposed in German and English, and there are slots for longer tutorials as well.

The range of possible topics is broad, so if you think you have anything interesting to share with a very passionate and technical audience of sysadmins and developers, here are some suggestions:

  • Operating Systems/Applications: architectures, privilege concepts, new developments, administration, mobile systems
  • Relevant new OS Kernel
  [Read more...]
Using MASTER_GTID_WAIT() to avoid stale reads from slaves in replication
+2 Vote Up -0Vote Down

I have just implemented MASTER_GTID_WAIT() in MariaDB 10.0. This can be used to give a very elegant solution to the problem of stale reads in replication read-scaleout, without incuring the overheads normally associated with synchronous replication techniques. This idea came up recently in a discussion with Stephane Varoqui, and is similar to the concept of Lamport logical clock described in this Wikipedia article.

I wanted to describe this, hoping to induce people to test and maybe start using this, as it is a simple but very neat idea, actually.

A very typical use of MariaDB/MySQL replication is

  [Read more...]
Correcting mysql.sock connection error
+0 Vote Up -0Vote Down

I write this post over to have a quick access to this tutorial than to promote the blog. The problem For Several times this has happened to me, always install a new server, or personal machine. I install mysql ,… Continue Reading →

Continue reading Correcting mysql.sock connection error

Related posts:
  • Corrigindo erro de conexão com mysql.sock
  • zend framework 2 with lighttpd
  • Main Subversion commands
  • More on 40% better single-threaded performance in MariaDB
    +1 Vote Up -0Vote Down

    In my previous post I wrote about how I achived a >40% speedup on sysbench read-only using profile-guided optimisation (PGO). While this is a preliminary result, I though it was so interesting that it deserved early mention. The fact that any benchmark can be improved that much shows clearly that PGO is something worth looking into. Even if we will probably not improve all workloads by 40%, it seems highly likely that we can obtain significant gains also for many real workloads.

    I had one or two interesting comments on the post that raise valid concerns, so I wanted to write a follow-up here, explaining some of the points in more details and going deeper into the performance counter measurements. As I wrote before, actual observations and measurements are crucial to fully understand performance of

      [Read more...]
    40% better single-threaded performance in MariaDB
    +5 Vote Up -0Vote Down

    Continuing my investigation of single-threaded performance in the MariaDB server, I managed to increase throughput of single-threaded read-only sysbench by more than 40% so far:

    I use read-only sysbench 0.4.12 run like this:

        sysbench --num-threads=1 --test=oltp --oltp-test-mode=simple --oltp-read-only --oltp-skip-trx run

    And mysqld is run with minimal options:

        sql/mysqld --no-defaults --basedir=X --datadir=Y --innodb-buffer-pool-size=128M

    With modern high-performance CPUs, it is necessary to do detailed measurements using the built-in performance counters in order to get any kind of understanding of how an application performs and what the bottlenecks are. Forget about looking at the code and counting instructions or cycles as

      [Read more...]
    MySQL/MariaDB single-threaded performance regressions, and a lesson in thread synchronisation primit
    +5 Vote Up -1Vote Down

    I took a quick look at MariaDB 10.0 single-treaded performance (simple read-only sysbench). One thing immediately leaps to the eye, and I thought it worthy of mention. It contains an important lesson about the use of synchronisation primitives and in particular "atomic operations" in MariaDB (and MySQL).

    I am using the Linux perf tool on this sysbench command:

      sysbench --num-threads=1 --test=oltp --oltp-test-mode=simple --oltp-read-only --oltp-skip-trx
    Look at the top offender in the output from perf report:
      1,54%  mysqld  mysqld               [.] set_thread_state_v1
    The only thing this does is set a string for SHOW PROCESSLIST (and the like) about what the thread is doing. And we are spending a whopping 1.5% of the total time doing this.

    And why? That becomes clear when looking

      [Read more...]
    Connecting your Linux to a Cisco AnyConnect (SSL) – part 3
    +0 Vote Up -0Vote Down

    Hi there !


    This is the final part of using openconnect  - You can check the older ones below:




    As mentioned previously, we will be covering here:

    • How to create a script to monitor such VPN using ICMP, and restart that VPN if it is down


    I would say, this is very straight forward and does not require much knowledge, so we are going to follow the same procedure as part 2 – Show the script in

      [Read more...]
    Connecting your Linux to a Cisco AnyConnect (SSL) – Part 2
    +0 Vote Up -0Vote Down

    Hey you!

    As said in the part 1 of this article, I will be covering here:

    • How to create a openconnect init script

    So, concerning the init script I will be posting parts of the script first, and then will put a link for download at the end.

    First of all, we need the shebang (#!/bin/bash) and then global variables that will be used along the script:

    # Path variables
    # VPN Variables
      [Read more...]
    First steps with MariaDB Global Transaction ID
    +8 Vote Up -1Vote Down

    My previous writings were mostly teoretical, so I wanted to give a more practical example, showing the actual state of the current code. I also wanted to show how I have tried to make the feature fit well into the existing replication features, without requiring the user to enable lots of options or understand lots of restrictions before being able to use it.

    So let us start! We will build the code from lp:~maria-captains/maria/10.0-mdev26, which at the time of writing is at revision knielsen@knielsen-hq.org-20130214134205-403yjqvzva6xk52j.

    First, we start a master server on port 3310 and put a bit of data into it:

        server1> use test;

      [Read more...]
    More on global transaction ID in MariaDB
    +6 Vote Up -0Vote Down

    I got some very good comments/questions on my previous post on MariaDB global transaction ID, from Giuseppe and Robert (of Tungsten fame). I thought a follow-up post would be appropriate to answer and further elaborate on the comments, as the points they raise are very important and interesting.

    (It also gives me the opportunity to explain more deeply a lot of interesting design decisions that I left out in the first post for the sake of brevity and clarity.)

    On crash-safe slave

    One of the things I really wanted to improve with global transaction ID is to make the replication slaves more crash safe with respect to their current replication state. This state is mostly persistently stored information about which event(s) were last executed on the slave, so that after a server restart the slave will know from  [Read more...]
    Global transaction ID in MariaDB
    +5 Vote Up -0Vote Down

    The main goal of global transaction ID is to make it easy to promote a new master and switch all slaves over to continue replication from the new master. This is currently harder than it could be, since the current replication position for a slave is specified in coordinates that are specific to the current master, and it is not trivial to translate them into the corresponding coordinates on the new master. Global transaction ID solves this by annotating each event with the global transaction id which is unique and universal across the whole replication hierarchy.

    In addition, there are at least two other main goals for MariaDB global transaction ID:

  • Make it easy to setup global transaction ID, and easy to provision a new slave into an existing replication hierarchy.
  • Fully support
  •   [Read more...]
    Super Python: three applications involving IRC bot master, MySQL optimization, and Website stress testing.
    +1 Vote Up -0Vote Down

    In my ongoing efforts to migrate my fun side projects and coding experiments from SVN to Git I’ve come across some of my favorite Python based apps – which are all available in their respective repos on BitBucket, as follows:

    IRC Bot Commander

    • What it does: it’s an IRC bot that takes commands and does your bidding on whichever remote server the bot is installed on.
    • How it does it: the bot runs on whatever server you install it on, then it connects to the IRC server and channel you configured it to connect to and it waits for you to give it commands, then it execs the commands and returns the output to your IRC chat window.
      [Read more...]
    Simple jQuery: how to validate IPv4 addresses and netmasks
    +0 Vote Up -0Vote Down

    Unfortunately jQuery doesn’t come with default form validation to check for ip-addresses or subnet masking. So without a long winded explanation here’s the code. Just include this as a separate JS file like the rest of your page’s JS.

    // 'ipv4': IPv4 Address Validator
    $.validator.addMethod('ipv4', function(value) {
        var ipv4 = /^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$/;    
        return value.match(ipv4);
    }, 'Invalid IPv4 address');
    // 'netmask': IPv4 Netmask Validator
    $.validator.addMethod('netmask', function(value) {
        var mask = /^[1-2]{1}[2,4,5,9]{1}[0,2,4,5,8]{1}\.
        return value.match(mask);
    }, 'Invalid IPv4 netmask');

    You can use it like this.

      [Read more...]
    Bash scripting: ElasticSearch and Kibana init.d scripts
    +0 Vote Up -0Vote Down

    As a follow up to the previous post about logstash, here are a couple of related init scripts for anyone implementing the OpenSource Log Analytics setup that is explained over at divisionbyzero. These have been tested on CentOS 6.3 and are based on generic RC functions from Redhat so they will work with Redhat, CentOS, Fedora, Scientific Linux, etc.

      [Read more...]
    Simple MySQL: using TRIGGERs to keep datetime columns updated without direct SQL calls
    +0 Vote Up -0Vote Down

    If you’ve ever used non-opensource code, or applications that you don’t have complete control over, then you may have run into situations you need to alter data on a per-row basis but been unable to do so for lack of application SQL access. The solution to this type of problem is to use a MySQL TRIGGER, which allows us to execute arbitrary SQL commands when defined events occur. Why is this useful and how does it work? Well…

    For example, I have a freeRADIUS server that uses MySQL as a backend for the user authentication, and one of my server applications (HostBill) provides a freeRADIUS plugin that allows my users to manage their RADIUS accounts; however the default freeRADIUS schema lacks a DATETIME column on the user table. When a user is created (INSERT) or has their password changed (UPDATE)

      [Read more...]
    OpenCode: MySQL procedures + python + shell code repositories now public
    +1 Vote Up -0Vote Down

    I write a fair number of scripts on this site and have posted a lot of code over the years. Generally if I am not pasting the code to be viewed on the webpage then I link to a file that a user can download; which leads to a lot of mish-mash code that doesn’t have a home. I’ve always kept the code files in a private SVN repo over the years but have recently moved them all to BitBucket Git repositories. So here they are: lots of code samples and useful bits of programming to save time.

    Generic Shell Scripts: https://bitbucket.org/themattreid/generic-bash-scripts/src
    Generic Python Scripts: https://bitbucket.org/themattreid/generic-python-scripts/src
    Generic MySQL Stored Procs:

      [Read more...]
    Simple MySQL: Converting ANSI SQL to SQLite3
    +0 Vote Up -0Vote Down

    I was digging through some old project code and found this script. Sometimes one finds oneself in an odd situation and needs to convert regular SQL, say from a MySQL database dump, into SQLite3 format. There’s not too much else to say, but here is a script that helps with the process. It can likely be improved but this handles the items that came up during conversion on initial runs.

    # NAME: convert-mysql-to-sqlite3.sh
    # AUTHOR: Matt Reid
    # DATE: 2011-03-22
    if [ "x$1" == "x" ]; then
       echo "Usage: $0 "
    cat $1 |
    grep -v ' KEY "' |   
    grep -v ' UNIQUE KEY "' |
    grep -v ' PRIMARY KEY ' |
    sed '/^SET/d' |          
    sed 's/ unsigned / /g' | 
    sed 's/ auto_increment/ primary key autoincrement/g' |
    sed 's/ smallint([0-9]*) / integer /g' | 
    sed 's/ tinyint([0-9]*) / integer /g' |  
    sed 's/ int([0-9]*) / integer /g' |      
    sed 's/
      [Read more...]
    MySQL: a convenient stored procedure for memory usage reporting
    +1 Vote Up -0Vote Down

    If you’ve ever been troubleshooting on the MySQL command line and needed to quickly see how much memory is being used then you’ve probably noticed that there are no built in commands to give you this data. Unlike other enterprise databases MySQL doesn’t have a very robust management system built in to help make the DBA’s life easier. It doesn’t come with built in Stored Procedures to report on usage statistics or generate handy reports; so we have to code them and import them to MySQL — no relying on Oracle to help us out here.

    So, here’s a stored procedure that can be imported to MySQL and run whenever you need to see the memory usage statistics. Installation and usage info is built into the SP below. The SP can also be downloaded from the repo:

      [Read more...]
    Integer overflow
    +0 Vote Up -1Vote Down

    What do you think of this piece of C code?

      void foo(long v) {
        unsigned long u;
        unsigned sign;
        if (v < 0) {
          u = -v;
          sign = 1;
        } else {
          u = v;
          sign = 0;
    Seems pretty simple, right? Then what do you think of this output from MySQL:
      mysql> create table t1 (a bigint) as select '-9223372036854775807.5' as a;
      mysql> select * from t1;
      | a                    |
      | -'..--).0-*(+,))+(0( | 
    Yes, that is authentic output from older versions of MySQL. Not just the wrong number, the output is complete garbage! This is my all-time favorite MySQL bug#31799. It was caused by code like the above C snippet.

    So can you spot what is wrong with the code? Looks pretty simple, does it

      [Read more...]
    Even faster group commit!
    +11 Vote Up -0Vote Down

    I found time to continue my previous work on group commit for the binary log in MariaDB.

    In current code, a (group) commit to InnoDB does not less than three fsync() calls:

  • Once during InnoDB prepare, to make sure we can recover the transaction in InnoDB if we crash after writing it to the binlog.
  • Once after binlog write, to make sure we have the transaction in the binlog before we irrevocably commit it in InnoDB.
  • Once during InnoDB commit, to make sure we no longer need to scan the binlog after a crash to recover the transaction. Of
  •   [Read more...]
    TunnelMaker, a simple script to generate multi-hop SSH tunnels
    +0 Vote Up -0Vote Down

    SSH tunnels provide a very effective means to access remote services and applications. Not only does it provide encryption of data between hosts, but it allows you to route connections between a sequence of servers, thus chaining connections. A common use of this method is to provide encrypted connections to MySQL servers so that user accounts can be limited to only “localhost” privileges, yet accessed from remote workstations without having to run MySQL+SSL.

    The concept is simple, for example let’s say you have three servers: localhost (your workstation in America), a server in Europe, and a server in Japan. You want to access Apache running on port 80 on the Japan server but because of firewall restrictions you cannot access port 80 remotely, and to make things more difficult the Japan server only allows SSH connections from the Europe server’s IP. We can

      [Read more...]
    Tech Messages | 2012-05-11
    +0 Vote Up -0Vote Down

    A special extended edition of Tech Messages for 2011-09-21 through 2012-05-11:

    Fun with Bash :: one liners
    +1 Vote Up -0Vote Down

    Here are some quick and easy bash commands to solve every day problems I run into. Comment and leave some of your own if you like. I might update this post with new ones over time. These are just some common ones.

    Iterate through directory listing and remove the file extension from each file
    ls -1 | while read each; do new=`echo $each |sed 's/\(.*\)\..*/\1/'` && echo $new && mv "$each" "$new"; done
    Output relevant process info, and nothing else
    ps axo "user,pid,ppid,%cpu,%mem,tty,stime,state,command"| grep -v "grep" | grep $your-string-here
    Setup a SOCKS5 proxy on localhost port 5050, to tunnel all traffic through a destination server
    ssh -N -D 5050 username@destination_server'
    Setup a SOCKS5 proxy via a remote TOR connection, using local port 5050 and remote TOR port 9050
    ssh -L 5050: username@destination_server'
    Display text or code file
      [Read more...]
    Living in the Prove It Culture
    +0 Vote Up -2Vote Down
    Engineering cultures differ from shop to shop. I have been in the same culture for 13 years so I am not an expert on what all the different types are. Before that I was living in Dilbert world. The culture there was really weird. The ideas were never yours. It was always some need some way off person had. A DBA, a UI "expert" and some product manager would dictate what code you wrote. Creativity was stifled and met with resistance.

    I then moved to the early (1998) days of the web. It was a start up environment. In the beginning there were just two of us writing code. So, we thought everything we did was awesome. Then we added some more guys. Lucky for us we mostly hired well. The good hires where type A personalities that had skills we didn't have. They challenged us and we challenged them. On top of that, we had a CEO who had been a computer hacker in his teens. So, he had just enough knowledge to

      [Read more...]
    Making rpm builds a first class citizen: How?
    +0 Vote Up -1Vote Down

    In my previous post I explained why I believe the production of RPM and DEB packages should be more integrated with the rest of your development process. Now it's time to look into how you can put the RPM build scripts inside your main source code repository, and in particular how I did that to produce RPM packages for Drizzle.

    read more

    Making rpm builds a first class citizen: Why?
    +0 Vote Up -2Vote Down

    Last weekend I released rpm files for the latest Drizzle Fremont beta (announcement). As part of that work I've also integrated the spec file and other files used by the rpmbuild into the main Drizzle bzr repository (but not yet merged into trunk). In this post I want to explain why I think this is a good thing, and in a follow up post I'll go into what I needed to do to make it work.

    (And speaking of stuff you can download, phpMyAdmin 3.5.0-alpha1 now supports Drizzle!)

    read more

    Could closed core prove a more robust model than open core?
    +0 Vote Up -0Vote Down

    When participating recently in a sprint held at Google to document four free software projects, I thought about what might have prompted Google to invest in this effort. Their willingness to provide a hotel, work space, and food for some thirty participants, along with staff support all week long, demonstrates their commitment to nurturing open source.

    Google is one of several companies for which I'll coin the term "closed core." The code on which they build their business and make their money is secret. (And given the enormous infrastructure it takes to provide a search service, opening the source code wouldn't do much to stimulate competition, as I point out in a posting on O'Reilly's radar

      [Read more...]
    What's New in CFEngine 3: Making System Administration Even More Powerful
    +1 Vote Up -0Vote Down

    CFEngine is both the oldest and the newest of the popular tools for automating site administration. Mark Burgess invented it as a free software project in 1993, and years later, as deployments in the field outgrew its original design he gave it a complete rethink and developed the powerful concept of promise theory to make it modular and maintainable. In this guise as version 3, CFEngine stands along with two other pieces of free software, Puppet and Chef, as key parts of enterprise computing. Along the way, Burgess also started a commercial venture, CFEngine AS, that maintains both the open source and proprietary versions of CFEngine.

    Diego Zamboni has recently taken the position of Senior Security Advisor at CFEngine AS and is writing

      [Read more...]
    Developer Week in Review: These things always happen in threes
    +0 Vote Up -0Vote Down

    Fall is being coy this year in the Northeast. We've been having on and off spells of very mild, almost summer-like weather over the last few weeks. That trend seems to be finally ending, alas, as there is possible snow forecasted for the weekend in New Hampshire. As the old joke goes, if you don't like the weather here, just wait five minutes.

    The fall also brings hunting to the area. The annual moose season just concluded (you need to enter a special lottery to get a moose permit), but deer season is just about to open. My son and I won't be participating this year, but we recently purchased the appropriate tools of the trade, a shotgun to hunt in southern NH (where you can't hunt deer with a rifle) and a Mosin Nagant 91/30 for the rest of the state. The later is probably

      [Read more...]
    New algorithm for calculating 95 percentile
    +1 Vote Up -0Vote Down

    The 95 percentile for query response times is and old concept; Peter and Roland blogged about it in 2008. Since then, MySQL tools have calculated the 95 percentile by collecting all values, either exactly or approximately, and returning all_values[int(number_of_values * 0.95)] (that’s an extreme simplification). But recently I asked myself*: must we save all values? The answer is no. I created a new algorithm** for calculating the 95 percentile that is faster, more accurate, and saves only 100 values.***

    Firstly, my basis of comparison is the 95 percentile algo used by

      [Read more...]
    Showing entries 1 to 30 of 245 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.