Do you know if your production MySQL servers will come back up when restarted? A recent support episode illustrates a number of best practices. The task looked trivial: Update a production MySQL server (replication master) with a configuration tuned and tested on a development server. Clean shutdown, change configuration, restart. Unfortunately, the MySQL daemon did not just ‘come back’, leaving 2 sites offline. Thus begins an illuminating debugging story. First place to look is the daemon error log, which revealed that the server was segfaulting, seemingly at the end of or just after InnoDB recovery. Reverting to the previous configuration did not help, nor did changing the InnoDB recovery mode. Working with the client, we performed a failover to a replication slave, while I got a second opinion from a fellow engineer to work out what had gone wrong on the server. Since debug symbols weren’t shown in the stack trace, we needed …[Read more]
Only the other day I was talking with someone who does a lot of work on the shell command line, but hadn’t used the GNU screen tool, so I’d better scribble a post about it as I regard it as an absolute must-have for any remote work, for multiple reasons.
First of all, what screen does. You start screen inside a terminal session (local or SSH remote), and then you can create additional sessions though Ctrl-A C. The initial screen is number 0, the next one 1, and so on. You can switch between screens with Ctrl-A # where # is the screen number. This way, you can have multiple things going within a single ssh connection, very handy. But that’s not all!
If you get disconnected (it happens and you reconnect, your screen sessions will still be there, and running too. You can reattach with screen -r. To do a nice disconnect, you can do Ctrl-A D (detach) before …[Read more]
I was actually exploring inotify-tools for something else, but they can also be handy for seeing what goes on below a mysqld process. inotify hooks into the filesystem handlers, and sees which files are accessed. You can then set triggers, or just display a tally over a certain period.
It has been a standard Linux kernel module since 2.6.13
(2005, wow that’s a long time ago already) and can be
used through calls or the inotify-tools (commandline). So with
the instrumentation already in the kernel,
inotify-tools is all you need to get started.
# inotifywatch -v -t 20 -r /var/lib/mysql/* /var/lib/mysql/zabbix/* Establishing watches... Setting up watch(es) on /var/lib/mysql/mysql/user.frm OK, /var/lib/mysql/mysql/user.frm is now being watched. [...] Total of 212 watches. Finished establishing watches, now collecting …[Read more]
Peter Lieverdink (also known as cafuego on IRC/identi.ca, engineer on OurDelta builds and for Open Query) has co-authored a book that’s available since Monday. The title is Pro Linux System Administration published by Apress.
These days some people don’t want to bother with system administration, and either hire or outsource. Others want to find out more and do things themselves (home and small office use), and that’s the intended audience for this book.
I’m happy to note (this is internal Open Query happiness but I’m pleased to share) that so far we have a 100% renewal rate for our Proactive Services for MySQL subscriptions. Some of the early clients have grown in the initial period and are have now moved to a higher # of hours (this can also be changed upward during a term), which is of course excellent both for the clients and for us.
I was in eager anticipation of this time since the introduction of the concept late last year, as it is of course the essential proof of whether a subscription service actually works over time. Ideally, you’d want renewal to be a simple straightforward process, with the client having experienced the value of the service. This is relatively straightforward in this case, since it’s not an insurance, emergency or retainer type arrangement – the client actually gets benefits each and …[Read more]
There’s actually no single “correct” answer! It all depends on
- where in a stack the component lives;
- the state of the market for that component region;
- sometimes even geographic location of the user comes into play.
Yes, for OSS projects modularity is handy in terms of handling contributions, but modularity may not be the best way to deal with a problem in a certain market state and situation!
Research has shown (see, for example, “The Innovator’s Solution” by Clayton Christensen) that the “integrated” region over time actually shifts to a subcomponent of an original integrated component that has since gone modular. An interesting example of this for MySQL its pluggable storage engine interface since version 5.1. MySQL is more modular now, but individual storage engines are tightly integrated for performance reasons, and in some cases they are even proprietary. …[Read more]
Open Database Alliance formed. Oracle buys Virtual Iron. AccesStream reaches version 1.0. And more.
Follow 451 CAOS Links live @caostheory
I’ve just met a fork named Maria
MySQL founder Monty Widenius and Percona CEO Peter Zaitsev announced the launch of the Open Database Alliance - “a collection of companies working together to provide the software, support and services for MariaDB, an enterprise-grade, community-developed branch of MySQL.”
Keep it. Make sure it gets correctly positioned in the coming months.
It appears that with the Oracle acquisition, the reason-to-exist for Falcon is regarded as gone (a non-Oracle-owned InnoDB replacement), previously seen as a strategic imperative - much delayed though.
But look, each engine has unique architectural aspects and thus a niche where it does particularly well. Given that Falcon exists, I’d suggest to not just “ditch it” but have it live as one of the pluggables. What Oracle will do to it is unknown, but Sun/MySQL can make sure of this positioning by making sure in the coming months that Falcon works in 5.1 as a pluggable engine, perhaps also creating a separate bzr project/tree for it on Launchpad.
Then the good work can find its way into the real world, now.
This is about the ending of the sponsorship of the mysql.meetup.com user groups by Sun/MySQL and their suggested move to Facebook.
If people want to move to Facebook, that’s fine. For those who want to stay but don’t have the local funding, I have an offer for you. Contact Open Query, and we’ll sponsor your group for the coming months. This is not open-ended, I think a more permanent solution is important (moving, sponsorship, whatever) but I want to make the effort for the community to prevent any groups from disappearing now just because of this.
I was the one who originally set up the agreement with meetup.com when they first started charging for meetups (and it turned out to be a very good business model for them!). When that change was announced, quite a few meetup organisers were going to quit. The …[Read more]