Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 102 Next 30 Older Entries

Displaying posts with tag: mysql server (reset)

Recent MySQL Announced Releases
Employee_Team +1 Vote Up -0Vote Down

Recently a several releases of MySQL Server were announced to our mailing lists. Do not miss following news:

  [Read more...]
Percona Playback 0.6 for MySQL now available
+1 Vote Up -0Vote Down

Percona is glad to announce the release of Percona Playback 0.6 for MySQL on April 9, 2013. Downloads are available from our download site and Percona Software Repositories.

Percona Playback for MySQL is a tool for replaying the load of one database server to another. Currently it can read queries from MySQL query-log and MySQL tcpdump files and run them on other MySQL server. With Percona Playback you can measure how a server or

  [Read more...]
A Visual Guide to the MySQL Performance Schema
Employee +1 Vote Up -0Vote Down

If you haven’t explored the MySQL Performance Schema yet, this is a good place to start.  This is Performance Schema 101, a basic introduction to the MySQL 5.6 performance_schema, which records runtime statistics from the MySQL database. The performance_schema is intended to provide access to useful information about server execution while having minimal impact on server performance.  Performance_schema is the name of both the storage engine and the database itself, and it was initially implemented  in MySQL 5.5. In MySQL 5.6 the engineers added quite a bit of new instrumentation.

The performance_schema database uses views or temporary tables that actually use little to no persistent disk storage .Memory allocation is all done at server startup, so there is no ongoing memory reallocation or

  [Read more...]
The InnoDB Quick Reference Guide is now available
+3 Vote Up -0Vote Down

I’m pleased to announce that my first book, the InnoDB Quick Reference Guide, is now available from Packt Publishing and you can download it by clicking here. It covers the most common topics of InnoDB usage in the enterprise, including: general overview of its use and benefits, detailed explanation of seventeen static variables and seven dynamic variables, load testing methodology, maintenance and monitoring, as well as troubleshooting and useful analytics for the engine. The current version of MySQL ships with InnoDB as the default table engine, so whether you program your MySQL enabled applications with PHP, Python, Perl or otherwise, you’ll likely benefit from this

  [Read more...]
Super Python: three applications involving IRC bot master, MySQL optimization, and Website stress testing.
+1 Vote Up -0Vote Down

In my ongoing efforts to migrate my fun side projects and coding experiments from SVN to Git I’ve come across some of my favorite Python based apps – which are all available in their respective repos on BitBucket, as follows:

IRC Bot Commander

  • What it does: it’s an IRC bot that takes commands and does your bidding on whichever remote server the bot is installed on.
  • How it does it: the bot runs on whatever server you install it on, then it connects to the IRC server and channel you configured it to connect to and it waits for you to give it commands, then it execs the commands and returns the output to your IRC chat window.
  [Read more...]
Simple MySQL: using TRIGGERs to keep datetime columns updated without direct SQL calls
+0 Vote Up -0Vote Down

If you’ve ever used non-opensource code, or applications that you don’t have complete control over, then you may have run into situations you need to alter data on a per-row basis but been unable to do so for lack of application SQL access. The solution to this type of problem is to use a MySQL TRIGGER, which allows us to execute arbitrary SQL commands when defined events occur. Why is this useful and how does it work? Well…

For example, I have a freeRADIUS server that uses MySQL as a backend for the user authentication, and one of my server applications (HostBill) provides a freeRADIUS plugin that allows my users to manage their RADIUS accounts; however the default freeRADIUS schema lacks a DATETIME column on the user table. When a user is created (INSERT) or has their password changed (UPDATE)

  [Read more...]
OpenCode: MySQL procedures + python + shell code repositories now public
+1 Vote Up -0Vote Down

I write a fair number of scripts on this site and have posted a lot of code over the years. Generally if I am not pasting the code to be viewed on the webpage then I link to a file that a user can download; which leads to a lot of mish-mash code that doesn’t have a home. I’ve always kept the code files in a private SVN repo over the years but have recently moved them all to BitBucket Git repositories. So here they are: lots of code samples and useful bits of programming to save time.

Generic Shell Scripts: https://bitbucket.org/themattreid/generic-bash-scripts/src
Generic Python Scripts: https://bitbucket.org/themattreid/generic-python-scripts/src
Generic MySQL Stored Procs:

  [Read more...]
Simple MySQL: Converting ANSI SQL to SQLite3
+0 Vote Up -0Vote Down

I was digging through some old project code and found this script. Sometimes one finds oneself in an odd situation and needs to convert regular SQL, say from a MySQL database dump, into SQLite3 format. There’s not too much else to say, but here is a script that helps with the process. It can likely be improved but this handles the items that came up during conversion on initial runs.

#!/bin/sh
####
# NAME: convert-mysql-to-sqlite3.sh
# AUTHOR: Matt Reid
# DATE: 2011-03-22
# LICENSE: BSD
####
if [ "x$1" == "x" ]; then
   echo "Usage: $0 "
   exit 
fi 
cat $1 |
grep -v ' KEY "' |   
grep -v ' UNIQUE KEY "' |
grep -v ' PRIMARY KEY ' |
sed '/^SET/d' |          
sed 's/ unsigned / /g' | 
sed 's/ auto_increment/ primary key autoincrement/g' |
sed 's/ smallint([0-9]*) / integer /g' | 
sed 's/ tinyint([0-9]*) / integer /g' |  
sed 's/ int([0-9]*) / integer /g' |      
sed 's/
  [Read more...]
MySQL: a convenient stored procedure for memory usage reporting
+1 Vote Up -0Vote Down

If you’ve ever been troubleshooting on the MySQL command line and needed to quickly see how much memory is being used then you’ve probably noticed that there are no built in commands to give you this data. Unlike other enterprise databases MySQL doesn’t have a very robust management system built in to help make the DBA’s life easier. It doesn’t come with built in Stored Procedures to report on usage statistics or generate handy reports; so we have to code them and import them to MySQL — no relying on Oracle to help us out here.

So, here’s a stored procedure that can be imported to MySQL and run whenever you need to see the memory usage statistics. Installation and usage info is built into the SP below. The SP can also be downloaded from the repo:

  [Read more...]
TunnelMaker, a simple script to generate multi-hop SSH tunnels
+0 Vote Up -0Vote Down

SSH tunnels provide a very effective means to access remote services and applications. Not only does it provide encryption of data between hosts, but it allows you to route connections between a sequence of servers, thus chaining connections. A common use of this method is to provide encrypted connections to MySQL servers so that user accounts can be limited to only “localhost” privileges, yet accessed from remote workstations without having to run MySQL+SSL.

The concept is simple, for example let’s say you have three servers: localhost (your workstation in America), a server in Europe, and a server in Japan. You want to access Apache running on port 80 on the Japan server but because of firewall restrictions you cannot access port 80 remotely, and to make things more difficult the Japan server only allows SSH connections from the Europe server’s IP. We can

  [Read more...]
SAN vs Local-disk :: innodb_flush_method performance benchmarks
+1 Vote Up -0Vote Down

If you’ve been tuning your MySQL database and have wondered what effect the innodb_flush_method settings have on write performance, then this information might help. I’ve recently been doing a lot of baseline load tests to show performance differences between localdisk and the new SAN we’re deploying. Since we run InnoDB for everything in production, and writes are very heavy, I decided to run comparison tests between two identical servers to find the best setting for innodb_flush_method. We have the following specs for the hardware:

  • Dell R610
  • 24 core Intel Xeon X5670 @ 2.93ghz
  • 72GB ECC RAM
  • Brocade 825 HBA
  • Local disk: RAID-10 15K SAS Ext3 (ugh)
  • SAN: Oracle 7420 with four Intel Xeon X7550 @ 2.00GHz, 512GB RAM, 2TB read-cache(MLC-SSD), 36GB write cache (SLC-SSD), 3 disk shelves populated with 60x2TB 7200RM SATA
  [Read more...]
A New Platform Supported
+0 Vote Up -0Vote Down

Ever hear of Red Hat Enterprise Linux 6.0? It was released on November the 10th of 2010 . . just over a year ago. In the last couple of days Oracle released the latest version of MySQL Server (5.5.18). Along with the bug fixes, etc they released RPM packages that cover RH EL 6.

Finally. A year later.

Not one given to griping, but really..does it take that long to roll packages for the new version? There were no significant changes in the operating system..in fact the RH EL 5 packages worked on RHEL 6 from my (albeit) limited experience with the combination.

I don't jump onto new versions of operating systems as soon as they come out. I prefer to let others be my beta testers before I put something into production. However, waiting a year seems a bit extreme for this release.

Even so, it's out now so enjoy! Now we have no excuse for not deploying 5.5.

km

 

 

 

Aloha – MySQL Dives into the Thread Pool
Employee +0 Vote Up -0Vote Down

By now you have probably heard about the MySQL thread pool plugin and API, but you may not have fully processed the details. Here’s the quick summary:  With the new thread pool plugin, there is now an alternative way to handle connection threads in MySQL Enterprise Edition.  With the plugin, MySQL connection threads are shared like an extraordinarily well managed timeshare in Hawaii.  When one connection is “idle”, asking nothing of and expecting nothing from the database, another connection can use that same thread for its database requests.  Threads are released by each connection as soon as the request is completed and  go back into the pool for re-use – just like the theoretical timeshare is up for grabs on the weeks you are not there.

In the older, and still default connection thread model, threads are dedicated to a single

  [Read more...]
MySQL Enterprise Backup 3.6 - New backup streaming, integration with Oracle Secure Backup and other common backup media solutions
Employee_Team +5 Vote Up -0Vote Down
All DBAs understand the importance and priority of quick, reliable database backup and recovery operations.  In fact, dating back to my early days with MySQL, the most commonly requested product features from the MySQL user base have been around online, non-blocking backup solutions for running MySQL servers.  In response, Oracle now provides MySQL Enterprise Backup (http://mysql.com/products/enterprise/backup.html) ("MEB") which performs high performant, online "hot" backups for MySQL databases.  MEB provides all of the backup/recovery features and functionality DBAs expect, all from a scriptable command line interface.  You can learn all about MEB in the related MySQL docs.

My congratulations and appreciation go out to Lars Thalmann and the MySQL Enterprise

  [Read more...]
Quick How-To for DRBD + MySQL + LVS
+0 Vote Up -0Vote Down

I wrote this up a while ago and decided that I didn’t want to lose it in a shuffle of documents during my transition to a new workstation. It’s the basics of setting up Heartbeat (LVS) + DRBD (block replication between active/passive master servers) + MySQL. This should give you the basics of a H/A system without the benefits of SAN but also without the associated cost. The validity of this setup for H/A purposes is highly dependent on your workload and environment. You should know the ins and outs of your H/A solution before deciding to blame the system for not performing as expected. As with all production systems you should test, test, test and test some more before going live.

When I get around to it later I’ll post my How-To for setting up RHCS + SAN + MySQL. You can download the DRBD document PDF here:

  [Read more...]
Quadrant Framework – rev7 update adds DyGraphs support
+0 Vote Up -0Vote Down

Quick update to the framework that was released yesterday; I’ve added automatic graph generation. I chose DyGraphs due to the quick ability to enable support – the HTML is very quick and simply loads the CSV data. It has the same zooming features of Highcharts without the JS overhead.

Now when you run a load test you will get (in the output directory) a mixture of files: the main cumulative CSV and HTML file for the hostname that was tested, and then one CSV and HTML per report variable that was tested. This means you don’t have to drag the main CSV file into an alternate program or spend time parsing out certain variables one at a time to generate specific graphs.  I’ve also added support for limiting output of SNMP variables (LOAD,CPU,MEM). Head over here and download the update: 

  [Read more...]
MySQL Load Testing Framework – initial release
+0 Vote Up -0Vote Down

It seems that everyone loves load testing these days. Problem is that everyone is using their own quick scripts, simple or complex, to drive their tests without the ability for other DBAs to duplicate those tests. Let’s say I write a great test and share my results and graphs on the blog – you want to run the same tests to see how your new DB servers compare in performance: this framework allows you to do that without duplicating any work or writing code. This is a basic release that will get the ball rolling. I’ve included some sample tests in the README file, so give them a try.

This codebase offers a user friendly framework for creating and visualizing MySQL database load test jobs. It is based around Sysbench, which is generally considered the industry standard load test application. The framework allows you to do the following:

    standardize your tests
  [Read more...]
MySQL Community – what do you want in a load testing framework?
+0 Vote Up -0Vote Down

So I’ve been doing a fair number of automated load tests these past six months. Primarily with Sysbench, which is a fine, fine tool. First I started using some simple bash based loop controls to automate my overnight testing, but as usually happens with shell scripts they grew unwieldy and I rewrote them in python. Now I have some flexible and easily configurable code for sysbench based MySQL benchmarking to offer the community. I’ve always been a fan of giving back to such a helpful group of people – you’ll never hear me complain about “my time isn’t free”. So, let me know what you want in an ideal testing environment (from a load testing framework automation standpoint) and I’ll integrate it into my existing framework and then release it via the BSD license. The main goal here is to have a standardized modular framework, based on sysbench,

  [Read more...]
Quick Reminder: MySQL 5.6.2 Dev Milestone and New MySQL Labs Features
Employee_Team +3 Vote Up -0Vote Down
As you know, Tomas announced the availability of MySQL 5.6.2 Dev Milestone in his MySQL O'Reilly and Collaborate keynote presentations a few weeks back.  Downloads so far have been great, which reflects the feedback and excitement we picked up on from the attendees of both events, especially at the O'Reilly Community reception on that Wednesday (it was great to so many friends, family, peers there!).

You can read John Russell's excellent recap of the new things in 5.6.2 and new innovations currently available in the MySQL Labs here.  And all of the 5.6 ready docs are available to help you learn how to enable and implement those you might find most interesting.

I am



  [Read more...]
MySQL Development Milestone 5.6.2: Taking MySQL Replication to the Next Level
Employee_Team +6 Vote Up -0Vote Down
The announcements at the O'Reilly MySQL and Oracle Collaborate conferences last week mark an exciting milestone in the development of MySQL replication.  The purpose of this blog is to summarize those announcements and provide links to further reading, published directly by the engineers who have been working hard to take MySQL replication to the next level.

A number of significant replication enhancements were released as part of the MySQL 5.5 GA just a few months ago. 
We are always listening to our customers and the community.  And, based on their input, the MySQL engineering team has continued to rapidly evolve


  [Read more...]
Top Features in MySQL 5.6.2 Development Milestone Release
Employee_Team +2 Vote Up -0Vote Down
Great news was announced today at the Collaborate 11 conference today;  the first development milestone release of MySQL 5.6.2 is now available to the MySQL Community!  This is significant because 5.6.2 builds on the momentum generated by the performance, scalability, InnoDB and Replication improvements that rolled out in version 5.5 just last Dec.  It also marks somewhat of a paradigm shift in how the MySQL Engineering team delivers new, defined sets of stable, development complete features (development "milestones") and cutting edge, currently under development, delivered early and often features via http://labs.mysql.com/.  Both options provide a means for Community users to begin using new versions and, more importantly, new features of MySQL very early on ensuring the final product has undergone the highest degree of testing,  [Read more...]
MySQL and RAMdisk, or how to make tmpdir-usage queries faster
+1 Vote Up -0Vote Down

Have you ever looked at your processlist and seen queries in the state “copying to tmp table” and then run an explain on it and noticed that the tmp table is being created on disk? Happens a lot with some servers and some workloads. Of course disk is much slower than RAM so this becomes a slow process and makes queries execute slower than they could if they were allowed to use RAM. So, one way to get this process to speed up (aside from tuning your queries which should be done first) is to create a tmpfs or ram-disk and let MySQL use that for it’s temp-table-on-disk creations. MySQL on Linux defaults to /tmp for the tmpdir location so this will need to be changed.

Here is how you get MySQL to use a 1G size tmpfs. How you size your tmpfs depends how much ram your system has and how much tmpdir space mysql needs for your workload. If you need more tmpdir space than

  [Read more...]
MySQL Analytics: updated query for table engine data statistics
+1 Vote Up -0Vote Down

This is a follow up to my previous post titled “MySQL analytics: information_schema polling for table engine percentages”. Here’s an updated query with more output and quicker execution time. What you get: innodb table space utilization percentage, data+index usage total and per innodb/myisam engine, innodb data/index/percentage, myisam data/index/percentages, and overall percentage values. Rather useful for profiling your table engine usage.

Sample output:
innodb_tablespace_utilization_perc: 100
total_size_gb: 26.275011910126
index_size_gb: 2.994891166687
data_size_gb: 23.280120743439
innodb_total_size_gb: 6.751220703125
innodb_data_size_gb: 5.2576751708984
innodb_index_size_gb: 1.4935455322266
myisam_total_size_gb: 19.523791207001









  [Read more...]
Custom MySQL config files to ensure maximum performance
+0 Vote Up -1Vote Down

The config files that come with MySQL server are generally not that good. They almost never work well for enterprise server loads and will leave most people wondering why the database needs someone to come fix it. In fact that might be why they roll those configs by default, to keep the consulting services alive and profitable. Personally I have no issues with consulting services, but everyone should start out with good configs to begin with. Well, no worries… here are config files for different system RAM configurations. Just choose the one that fits how much RAM your server has and make sure the cnf file is in place before you install the MySQL server RPM/tar/deb packages. Obviously you’ll want to remove the extension of the filename so it reads “my.cnf” and not “my.cnf_64GB” or the like. These cnfs have

  [Read more...]
A List of Useful MySQL Tuning Equations
+5 Vote Up -0Vote Down

It’s always good to have some equations for reference when you are tuning a MySQL server. How else will you know what to set your buffer sizes to after all? If you have some that I’ve missed… add a comment!

Per-Thread Buffer memory utilization (read_buffer_size + read_rnd_buffer_size + sort_buffer_size + thread_stack + join_buffer_size + binlog_cache_size) * max_connections Global Buffer memory utilization innodb_buffer_pool_size + innodb_additional_mem_pool_size + innodb_log_buffer_size + key_buffer_size + query_cache_size Threads and Connections thread_cache miss rate = Threads_created / Connections connection ratio = (max_used_connections*100)/ max_connections threads_per_second = threads_created / uptime Key Buffer key_buffer_free = (key_blocks_unused *  [Read more...]
Win a free book at the February Python Book Contest
+2 Vote Up -0Vote Down

This month is a special month. It’s not because of Valentines day or even the exciting day where we see groundhogs. No, this month is special because I’m have a book contest where you, the reader, get to win something free for doing absolutely nothing more than posting a comment saying that you want one of the several books I have available in the contest.

So without getting into boring details I’ll keep this short. I’ve been reviewing a lot of books lately and I think it’s time to get some books into people’s hands to enjoy themselves. This month the giveaways are all Python oriented.

So, all you have to do is take a look at the following titles and post a comment here saying that you want one of them. At the end of the month two readers will be chosen via a random list sorting python script I’ve whipped up for just this purpose. You

  [Read more...]
Python for Automation: using pdsh for a menu-driven command execution environment
+1 Vote Up -0Vote Down

I’ve been playing around with some quick system automation scripts that are handy to use when you don’t want / need to setup a chef or puppet action. I like to keep all of my hostnames and login details in a MySQL database (a cmdb actually) but for this example we’ll just use a couple of nested lists. This script executes commands in parallel across the hosts you choose in the menu system via the “pdsh” command, so make sure you have that installed before running. Alternately you can change the command call to use ssh instead of pdsh for a serialized execution, but that’s not as fun or fast. With some customizations here and there you can expand this to operate parallelized jobs for simplifying daily work in database administration, usage reporting, log file parsing, or other system automation as you see fit. Here’s the code. Comments welcome as

  [Read more...]
Yet Again On Subqueries
+4 Vote Up -1Vote Down

…with a bit of strategic thinking

They come back, every now and then. Subqueries are far from being perfect at MySQL and they can give you some serious headaches.

Skilled MySQL developers know it better. They avoid subqueries as much as they can. It is not that subqueries do not work, it is just that the optimizer sometimes is, well, “not that optimised”.

So you may stay away from subqueries with some good SQL review. But what happens when the subquery is automatically generated by a script or a tool? If you can change the statement, I’m afraid you need to find some serious workarounds that vary case by case.

Here is an example that I found few weeks ago when I visited one of our customers.

Our customer used Magento for its site. Magento used a couple of queries that I will

  [Read more...]
Comparing InnoDB to MyISAM Performance
Employee_Team +6 Vote Up -0Vote Down
The MySQL performance team in Oracle has recently completed a series of benchmarks comparing Read / Write and Read-Only performance of MySQL 5.5 with the InnoDB and MyISAM storage engines.

Compared to MyISAM, InnoDB delivered 35x higher throughput on the Read / Write test and 5x higher throughput on the Read-Only test, with 90% scalability across 36 CPU cores.

A full analysis of results and MySQL configuration parameters are documented in a new whitepaper (http://www.mysql.com/why-mysql/white-papers/mysql_5.5_perf_myisam_innodb.php)


In addition to the benchmark, the new whitepaper (http://www.mysql.com/why-mysql/white-papers/mysql_5.5_perf_myisam_innodb.php), also includes:
- A discussion of the use-cases for each storage engine
- Best practices for users








  [Read more...]
Review: MySQL for Python by Albert Lukaszewski
+1 Vote Up -0Vote Down

Packt Publishing recently sent me a copy of MySQL for Python to review and after reading through the book I must say that I’m rather impressed at the variety of topics that the book covers.

It starts off with the basics of setting up MySQL for your testing/development needs by going over several of the common installation and configuration methods. After that it’s a quick intro for connection methods and simple error reporting for connections. The author gives a quick intro to CRUD and how it relates to databases and python before heading into the common tasks of simple queries. I was surprised to see some database profiling discussion; which is rather handy for a new coder or a

  [Read more...]
Showing entries 1 to 30 of 102 Next 30 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.