Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 26

Displaying posts with tag: troubleshooting (reset)

By all means, learn from my mistakes as a DBA!
+1 Vote Up -0Vote Down
Here are 3 recent ' oops... wish I hadn't done that :/ ' mistakes I've made since joining moz that you might as well avoid (I'm sure there will be more, but they better not be the same) Reviewing config files for MySQL, but not all of the defaults      We recently migrated a few MySQL databases to a new datacenter, and took advantage of the migration to upgrade the MySQL version(s) at the
The network is reliable
+2 Vote Up -0Vote Down

A fascinating post-mortem on high profile network failures:

This post is meant as a reference point–to illustrate that, according to a wide range of accounts, partitions occur in many real-world environments. Processes, servers, NICs, switches, local and wide area networks can all fail, and the resulting economic consequences are real. Network outages can suddenly arise in systems that are stable for months at a time, during routine upgrades, or as a result of emergency maintenance. The consequences of these outages range from increased latency and temporary unavailability to inconsistency, corruption, and data loss. Split-brain is not an academic concern: it happens to all kinds of systems–sometimes for days on end. Partitions deserve serious consideration.

MySQL binlogs - Don't forget to do your homework!
+0 Vote Up -1Vote Down
Now that I'm back doing just database stuff, I've come to realize I've gotten a little sloppy about doing my homework.  Homework's never been my favorite thing in the world, but it often reduces stress when your under the gun during an outage or upgrade... We had a MySQL database server that's been slow on DML changes, and based on the slowest statements being 'COMMIT', we had a good mind
Troubleshooting Performance Diagrams
Employee_Team +1 Vote Up -0Vote Down


Last year, when I was speaking about MySQL performance at Devconf in Moscow, I expected my audience will be very experienced as this always happen at all PHPClub conferences. So I had to choose: either make full-day seminar and explain people every basic of performance, or rely on their knowledge and make some one and half hours seminar. I prefer short speeches, so I considered latter.



But even with such a mature audience you don't always know if they knew some or another basic thing. Like somebody can be good analyzing EXPLAIN output and other is in reading InnoDB Monitor printout. Also, native language of the audience is not English and it would be always good to have short reference to simple things, described in their native




  [Read more...]
Do we need a MySQL Cookbook?
+2 Vote Up -2Vote Down

The blog title says it all: Do we need a MySQL Cookbook? I tend to think so.

This seems to be something that is missing with current MySQL documentation. There is lots of information available but finding the appropriate bit can be quite tedious and it often requires looking in multiple places.

A lot of other software has such books, but for some reason MySQL seems to be missing one.

A recent example comes from a “documentation feature request” I posted today: http://bugs.mysql.com/bug.php?id=68171. MySQL 5.6 provides a way to “move InnoDB tables” from one server to another. There are many reasons why you may want to do it, but the documentation is currently rather sparse. A simple “example recipe” for this would be good, as would an equivalent recipe for other

  [Read more...]
MySQL Cluster: Troubleshooting Error 157 / 4009 Cluster Failure
+1 Vote Up -0Vote Down
0 0 1 519 2962 Severalnines AB 24 6 3475 14.0 Normal 0 false false false false EN-US JA X-NONE
Suddenly your application starts throwing "error 157" and performance degrades or is non-existing. It is easy to panic then and try all sorts of actions to get past the problem. We have seen several users doing:
  • rolling restart
  • stop cluster / start cluster
because they also see this in the error logs:120828 13:15:11 [Warning] NDB: Could not acquire global schema
  [Read more...]
A CTO Must Never Do This…
+0 Vote Up -4Vote Down

Read the original article at A CTO Must Never Do This…

A couple years back I was contacted to look at a very strange problem.

The firm ran flash sales. An email goes out at noon, the website traffic explodes for a couple of hours, then settles back down to a trickle.

Of course you might imagine where this is going. During that peak, the MySQL database was brought to its knees. I was asked to do analysis during this peak load, and identify and fix problems. Make it go faster, please!

First day on the job I’m working with a team of outsourced DBAs. I was also working

  [Read more...]
MySQL 5.6 too verbose when creating data directory
+4 Vote Up -0Vote Down

When I install a MySQL package using MySQL Sandbox, if everything goes smoothly, I get an informative message on standard output, and I keep working.

This is OK


$HOME/opt/mysql/5.5.15/scripts/mysql_install_db --no-defaults \
--user=$USER --basedir=$HOME/opt/mysql/5.5.15 \
--datadir=$HOME/sandboxes/msb_5_5_15/data \
--lower_case_table_names=2
Installing MySQL system tables...
OK
Filling help tables...
OK

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:

/Users/gmax/opt/mysql/5.5.15/bin/mysqladmin -u root password 'new-password'















  [Read more...]
InnoDB disabled if ib_logfile files corrupted
+1 Vote Up -0Vote Down


I recently came across a dev VM running MySQL 5.0.77 (an old release, 28 January 2009) that didn’t have InnoDB available. skip-innodb wasn’t set, SHOW VARIABLES LIKE '%innodb%' looked as expected, but with one exception: the value of have-innodb was DISABLED.

I confirmed this with SHOW ENGINES:

(root@localhost) [(none)]> show engines;
+------------+----------+----------------------------------------------------------------+
| Engine     | Support  | Comment                                                        |
+------------+----------+----------------------------------------------------------------+
| MyISAM     | DEFAULT  | Default engine as of MySQL 3.23 with great performance

  [Read more...]
Inode allocation on Amazon AWS RDS (Relation Database Service)
+5 Vote Up -0Vote Down

The attached storage used by Amazon’s managed Relational Database Service has a known issue where the bytes per inode ratio is very high (default on RHEL5 systems is 4096, to be found in /etc/mke2fs.conf). Amazon does not allow any administrative access to these instances so there is no way to reformat the filesystem to allocate more inodes, or attach storage the user can format with a different ratio.

This becomes problematic for databases that have many small tables (generally MyISAM tables, or InnoDB with the innodb_file_per_table setting) which quickly consume the available inodes. When the inode allocation is exhausted MySQL responds with

"ERROR 1030 (HY000): Got
  [Read more...]
MySQL performance flow chart
+0 Vote Up -0Vote Down

Here’s an old-but-still-relevant (re)post from Major Hayen on improving MySQL performance. It’s a neat, concise reference guide for MySQL emergencies!

Original post

mount: /dev/sdb1 already mounted or /mysql busy
+0 Vote Up -0Vote Down

We added a 500GB 7.2K SATA/300 Hitachi Deskstar E7K500 16MB disk to one of our dev servers and partitioned using fdisk and formatted the partition with etx3. When we tried mounting the same, we got the following error :

[root@xyz user]# mount -t ext3  /dev/sdb1 /mysql
mount: /dev/sdb1 already mounted or /mysql busy

lsof didn’t provide any open files that might be linked to this problem or there was any “famd” running. Finally doing the following steps to remove the logical devices from the device-mapper driver helped us fix the problem.

[root@xyz user]# dmsetup ls
ddf1_44656c6c202020201028001510281f033832b7a2f6678dab   (253, 0)
ddf1_44656c6c202020201028001510281f033832b7a2f6678dab1  (253, 1)

[root@xyz user]# dmsetup remove




  [Read more...]
LOAD DATA: a tricky replication issue
+7 Vote Up -0Vote Down
When you are importing large amounts of data from other sources LOAD DATA is a common method of inserting data into a table.
It is one of the old commands implemented in MySQL. As such it is very fast, and it has been optimized for both MyISAM and InnoDB.
All is well when you are loading data into a standalone server. All is almost well when you are using replication. LOAD DATA used to be a problem in old versions of MYSQL, prior to 4.1. With recent versions of MySQL, it is replicated correctly, and sometimes efficiently.
The trouble starts when the data file is big. The definition of big and the amount of trouble that you can get depends on many factors. That's why users may not realize that this problem exists, even with fairly large files, and then being hit by this


  [Read more...]
MySQL Remote Connections for Replication
+1 Vote Up -0Vote Down
Here is another interesting problem:When trying to set up master to master replication from Server A to Server B, the replication user can login from A to B, but not the other way. The error thrown out by MySQL:ERROR 1130 (HY000): Host '' is not allowed to connect to this MySQL serverThings to check:1. Replication user password and host2. Ability to ping the remote server3. nslookup the remote
Lost and Found ?
+0 Vote Up -0Vote Down
Sometimes you just have to laugh at the crazy things that can kill a good evening. I had this brilliant idea to change our replication setup on one of our Master-Master replication server setups this week. I got sick of having to restart MySQL every time we wanted to add a new database and have it included in the list of replicated databases - we were using replicate-do-db in our configs.So it
What just happened to the database?
+0 Vote Up -0Vote Down
It's always fun when you come into work and notice that one of your database monitors/graphs has changed from showing almost no row accesses/sec to 40,000 rows/sec in a matter of minutes. And then by lunch time they are showing 90,000 rows/sec. What's up with that?MySQL Enterprise Monitor Row Accesses:Well, of course the first thing you do is check all your cacti monitors, because there is
Spinning the Wheel of Protocols
+0 Vote Up -0Vote Down

Wheel of protocol, turn turn turn.
Tell us the lesson that we should learn.
(with apologies to the original source)

Writing a book comes with many challenges. For me, writing a MySQL book for MySQL beginners, who may or may not be database beginners, has fed my compulsion to research and test bizarre interactions.

Today’s lesson is on what protocol is used when connecting to a local mysqld instance on a non-Windows machine. The TCP/IP protocol is used by default when connecting on a Windows machine, and connecting from any operating system to a non-local mysqld instance. I am assuming the connections are being made by a command line client such as mysql, mysqladmin or


  [Read more...]
Frustration with Community vs Enteprise
+0 Vote Up -0Vote Down

I was working on a client’s server today to troubleshoot some variances between the result timing of some queries. Guess what I came across - the profiler is not available in certain enterprise releases but it is available on community versions of the same release number.

I can understand if that feature was something that wasn’t fully tested in the enterprise code base and thus was only released in the community version - but if that’s the case then I don’t understand why the same version releases of Community and Enterprise can have different feature sets. That goes against the whole idea of versioning. Someone correct me if I’m wrong here but that is very frustrating.

MySQL Replication and bad assumptions
+0 Vote Up -0Vote Down
Sometimes I amaze myself in my capacity to make assumptions about how things should work, especially when it comes to test plans... ( You know what happens when we assume, right? )I had this great idea to setup a couple slaves off a master-master replication set something like this:MASTER A <--------------> MASTER B | | | | |
MySQL Data Woes... or, Making Use of the Information_Schema
+0 Vote Up -0Vote Down
To be fair - I highly doubt it has anything to do with MySQL.Database corruption - that's a different issue, often related to a bug in database code; but this was a data corruption issue, which is always a pain in the you know what to figure out and get fixed.I spent the last day trying to figure out the weirdness in a couple of our databases, digging around, explaining to developers that
MySQL: Improving your skills with Forums
+0 Vote Up -0Vote Down
I haven't been a big user of the MySQL Forums till recently.I'm not sure why - partly because I like to think I know something about what I do ( I just need to remember that there is always someone else who knows more), partly because I have a need to re-invent the wheel as often as possible, and partly because I dont' like waiting around for an answer...It can be a little overwhelming when you
MySQL: Getting Creative with Partitioning
+0 Vote Up -0Vote Down
Lately, I've been trying to keep up with at least one of the MySQL Forums: Partitioning.It's a great way to keep on top of issues surrounding partitioning, and also get an idea of what people are trying to do with the new 5.1 feature. Richard came up with an interesting problem that I jumped into only to realize that I hadn't done my homework, and my initial suggestion wouldn't work at all due
MySQL: Finally an ability to trace/profile
+0 Vote Up -0Vote Down
Finally! The ability to look a little closer into what's happening with SHOW PROFILEHere’s how it works:mysql> set profiling=1;mysql> select count(*) from mysql.user;+----------+| count(*) |+----------+| 5 |+----------+1 row in set (0.00 sec)mysql> show profile;+--------------------------------+----------+| Status | Duration |+--------------------------------+-----
Best Practices - Oops...
+0 Vote Up -0Vote Down
Yea, yea, yea... best practices.I spent about a week troubleshooting issues on a MySQL 5.1 replication issue where certain transactions causing duplicate key on index errors stopped replication. Easy enough to fix, right? mysql> SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1; mysql> start slave;Hmm... nice until it happens almost every minute...So, after reading every link I could find on the internet,
MySQL : the beauty in beta
+0 Vote Up -0Vote Down
Beta or not, here we come. Yea, you'd think I'd learn to check out the known bugs list before letting developers/qa have at the new database servers running in the lab with MySQL 5.1.21-b . I mean, how bad could beta really be?Well, as we should all know, beta is called beta for a reason, but sometimes we open source fanatics tend to jump on board, dealing with issues along the way. This
Why MySQL says the server is not configured as a slave
+0 Vote Up -0Vote Down

Is MySQL giving you the error message "ERROR 1200 (HY000): The server is not configured as slave; fix in config file or with CHANGE MASTER TO" when you try to run START SLAVE? There are a few simple troubleshooting steps to take, but I always forget what to do. This article is to help me remember in the future!

Showing entries 1 to 26

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.