Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 90 Next 30 Older Entries

Displaying posts with tag: Scripting (reset)

Using mysqldump and the MySQL binary log – a quick guide on how to backup and restore MySQL databases
+0 Vote Up -0Vote Down

Be sure to check out my other posts on mysqldump:
- Scripting Backups of MySQL with Perl via mysqldump
- Splitting a MySQL Dump File Into Smaller Files Via Perl
- Creating and restoring database backups with mysqldump and MySQL Enterprise Backup – Part 1 of 2
- Creating and restoring database backups with mysqldump and



  [Read more...]
Installing and testing the MySQL Enterprise Audit plugin
+0 Vote Up -0Vote Down

MySQL Enterprise Edition includes the most comprehensive set of advanced features, management tools and technical support to achieve the highest levels of MySQL scalability, security, reliability, and uptime. It reduces the risk, cost, and complexity in developing, deploying, and managing business-critical MySQL applications.

(from http://www.mysql.com/products/enterprise/)

MySQL Enterprise Audit provides an easy to use, policy-based auditing solution that helps organizations to implement stronger security controls and to satisfy regulatory compliance.

As more sensitive data is collected, stored and used online, database auditing becomes an essential component of any security strategy. To guard against

  [Read more...]
MySQL Enterprise Monitor – send advisor events to your chat client with Perl and Jabber
+0 Vote Up -0Vote Down

MySQL Enterprise Monitor (http://www.mysql.com/products/enterprise/monitor.html) (MEM) is part of the MySQL Enterprise Edition, and MEM provides real-time visibility into the performance and availability of all your MySQL databases. MEM and the MySQL Query Analyzer continuously monitor your databases and alerts you to potential problems before they impact your system. It’s like having a “Virtual DBA Assistant” at your side to recommend best practices to eliminate security vulnerabilities, improve replication, optimize performance and more. As a result, the productivity of your developers, DBAs and System Administrators is improved significantly.

With MEM, you have a couple of notification options for receiving information when MEM has

  [Read more...]
MySQL Replication – Creating a New Master/Slave Topology with or without Virtual Machines
+4 Vote Up -0Vote Down

In my last few posts, I wrote about “How to install MySQL replication using GTID’s” (Part One, Part Two). In this post, I will show you how to install MySQL 5.6 and set up replication between two MySQL servers the “old fashioned way” using the binary log and binary log position.

I am going to create some virtual machines instead of using individual servers. But, you can

  [Read more...]
Using MySQL Utilities Workbench Script mysqldbcompare To Compare Two Databases In Replication
+4 Vote Up -0Vote Down

In my last two posts, I wrote about setting up replication with MySQL 5.6 using Global Transaction Identifiers. Even when I set up replication “the old-fashioned way“, one thought always enters my mind – did all of the data copy over to the slave? And, even after the master/slave has been running for a while, I am always wondering if the data in the slave matches the master. Or did the change that I made to that table make it over to the slave? It is probably more of a case of paranoia on my part, as

  [Read more...]
MySQL Replication with Global Transaction Identifiers – Step-by-Step Install and Addition of Slaves – Part One
+3 Vote Up -1Vote Down

One of my favorite features of MySQL (http://mysql.com) is replication. Replication provides you with the ability to have MySQL automatically copy data from one MySQL instance to another. There are many benefits to using replication, but I just like having an extra copy of my data on another server in case the main server crashes. But if the master crashes, I can then use the MySQL mysqlfailover script to automatically failover from the master to the slave. (see my earlier post – Using the MySQL Script mysqlfailover for Automatic Failover

  [Read more...]
Retrieving List of MySQL Users and Grants with Perl
+0 Vote Up -0Vote Down

Before I upgrade MySQL to the latest and greatest version, one of the first things that I do is export the user and grant information. In the past, I would keep all of my user information (user name, password, grants) in a text file, with the SQL for each user/grant ready to be executed on the upgraded server. I did use my own form of “mental encryption” for my passwords, so the passwords weren’t in plain English. But then I would have to decode my passwords each time before I executed the SQL statements.

When I upgrade, I usually like to dump all of the data and import it into the new version, so I have a fresh copy of the database. The MySQL server that I have is for my personal use and the data size is relatively small, so for my case it doesn’t take long to import the data.

But there were times when I

  [Read more...]
MySQL Replication – Multi-Threaded Slaves (Parallel Event Execution)
+3 Vote Up -1Vote Down

If you aren’t familiar with MySQL replication, “Replication enables data from one MySQL database server (the master) to be replicated to one or more MySQL database servers (the slaves). Replication is asynchronous by default – slaves need not to connected permanently to receive updates from the master. This means that updates can occur over long-distance connections and even over temporary or intermittent connections such as a dial-up service. Depending on the configuration, you can replicate all databases, selected databases, or even selected tables within a database.” (From: http://dev.mysql.com/doc/refman/5.5/en/replication.html).

I use MySQL replication on my home office server. I don’t really have much data to store, but it

  [Read more...]
MySQL 5.6 Delayed Replication – Making a Slave Deliberately Lag Behind a Master
+2 Vote Up -0Vote Down

In the majority of MySQL replication scenarios, you want your slave databases to be a mirror of your master databases. You usually don’t want your slave to be behind your master by more than a few seconds – and your main goal is for your slave to always be in sync with your master. Would you ever want your slave to deliberately be a few seconds, minutes or even hours behind your master? There have been several suggestions from MySQL users over the years regarding this functionality as “feature request” (even though most of the requests were submitted as MySQL “bugs”, which was the easiest way to submit such a request).

The first request (that I could find) was by Jason Garrett, back in August of 2006, and was logged as “bug

  [Read more...]
Simple jQuery: how to validate IPv4 addresses and netmasks
+0 Vote Up -0Vote Down

Unfortunately jQuery doesn’t come with default form validation to check for ip-addresses or subnet masking. So without a long winded explanation here’s the code. Just include this as a separate JS file like the rest of your page’s JS.

// 'ipv4': IPv4 Address Validator
$.validator.addMethod('ipv4', function(value) {
    var ipv4 = /^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$/;    
    return value.match(ipv4);
}, 'Invalid IPv4 address');

// 'netmask': IPv4 Netmask Validator
$.validator.addMethod('netmask', function(value) {
    var mask = /^[1-2]{1}[2,4,5,9]{1}[0,2,4,5,8]{1}\.
[0-2]{1}[0,2,4,5,9]{1}[0,2,4,5,8]{1}\.
[0-2]{1}[0,2,4,5,9]{1}[0,2,4,5,8]{1}\.
[0-9]{1,3}$/;    
    return value.match(mask);
}, 'Invalid IPv4 netmask');

You can use it like this.

$("#myform_here").validate({
    rules:{
	ipaddress:{
  [Read more...]
Bash scripting: ElasticSearch and Kibana init.d scripts
+0 Vote Up -0Vote Down

As a follow up to the previous post about logstash, here are a couple of related init scripts for anyone implementing the OpenSource Log Analytics setup that is explained over at divisionbyzero. These have been tested on CentOS 6.3 and are based on generic RC functions from Redhat so they will work with Redhat, CentOS, Fedora, Scientific Linux, etc.

  [Read more...]
Using the MySQL Script mysqlfailover for Automatic Failover with MySQL 5.6 GTID Replication
+4 Vote Up -0Vote Down

This post is the second in a series that I will be doing on MySQL Workbench Utilities – Administer MySQL with Python Scripts. You may want to read the first half of this post to understand how MySQL Workbench Utilities work and how you access the scripts. These scripts were written by Chuck Bell (a MySQL (http://mysql.com) employee)

  [Read more...]
OpenCode: MySQL procedures + python + shell code repositories now public
+1 Vote Up -0Vote Down

I write a fair number of scripts on this site and have posted a lot of code over the years. Generally if I am not pasting the code to be viewed on the webpage then I link to a file that a user can download; which leads to a lot of mish-mash code that doesn’t have a home. I’ve always kept the code files in a private SVN repo over the years but have recently moved them all to BitBucket Git repositories. So here they are: lots of code samples and useful bits of programming to save time.

Generic Shell Scripts: https://bitbucket.org/themattreid/generic-bash-scripts/src
Generic Python Scripts: https://bitbucket.org/themattreid/generic-python-scripts/src
Generic MySQL Stored Procs:

  [Read more...]
WebEx Today! MySQL Cluster 7.3 Development Release: What’s New
+3 Vote Up -0Vote Down

Thursday, October 25, 2012
MySQL Cluster 7.3 Development Release: What’s New

Join the webinar to get all of the detail on MySQL Cluster 7.3 DMR and Early Access releases. Announced at the recent MySQL Connect conference, Oracle has published previews of MySQL Cluster 7.3.

MySQL Cluster 7.3 delivers more power and flexibility than ever before. The webinar will demonstrate each of these new features and show you how to get started.

WHO:
Andrew Morgan, MySQL Product Management
Mat Keep, MySQL Product Management

WHAT:
MySQL Cluster 7.3 Development Release: What’s New web presentation.

WHEN:
Thursday, October 25, 2012: 09:00 Pacific time (America)

Thu, Oct 25: 06:00 Hawaii time
Thu, Oct 25: 10:00 Mountain time (America)
Thu, Oct 25: 11:00






  [Read more...]
Simple MySQL: Converting ANSI SQL to SQLite3
+0 Vote Up -0Vote Down

I was digging through some old project code and found this script. Sometimes one finds oneself in an odd situation and needs to convert regular SQL, say from a MySQL database dump, into SQLite3 format. There’s not too much else to say, but here is a script that helps with the process. It can likely be improved but this handles the items that came up during conversion on initial runs.

#!/bin/sh
####
# NAME: convert-mysql-to-sqlite3.sh
# AUTHOR: Matt Reid
# DATE: 2011-03-22
# LICENSE: BSD
####
if [ "x$1" == "x" ]; then
   echo "Usage: $0 "
   exit 
fi 
cat $1 |
grep -v ' KEY "' |   
grep -v ' UNIQUE KEY "' |
grep -v ' PRIMARY KEY ' |
sed '/^SET/d' |          
sed 's/ unsigned / /g' | 
sed 's/ auto_increment/ primary key autoincrement/g' |
sed 's/ smallint([0-9]*) / integer /g' | 
sed 's/ tinyint([0-9]*) / integer /g' |  
sed 's/ int([0-9]*) / integer /g' |      
sed 's/
  [Read more...]
Top 10 MySQL Tips and Mistakes for PHP Developers (Webinar)
+1 Vote Up -0Vote Down

Tuesday, September 25, 2012

MySQL and PHP are two key components in the open-source LAMP stack, and are widely used by web developers. One day you launched a website, and it gradually gained traction and now serves hundreds times more traffic than day one. As a PHP developer, have you ever looked back and adjusted the MySQL configurations accordingly to meet the most current website load? Have you ever wondered if there are secret tips you might have missed or common mistakes that you could have avoided but never thought of? If you answered yes to either question, this presentation is for you!

In this live webinar, PHP & MySQL experts Ulf Wendel and Johannes Schlüter will share the top 10 tips and mistakes for PHP developers. After a quick overview of lesser known but cool client and server features and plugins, and a recap

  [Read more...]
MySQL Workbench Utilities – Clone MySQL users with mysqluserclone
+4 Vote Up -0Vote Down

This post is one in a series that I will be doing on MySQL Workbench Utilities – Administer MySQL with Python Scripts.

MySQL Utilities are a part MySQL Workbench. The utilities are written in Python, available under the GPLv2 license, and are extendable using the supplied library. They are designed to work with Python 2.x greater than 2.6. If you don’t have Workbench, you may download the MySQL Utility scripts from

  [Read more...]
Scripting continued
+0 Vote Up -0Vote Down

Since I have been discussing scripting lately I thought I would continue with another topic I touched on briefly - backups.

I have written and modified the following script over the last few years. I have used it (and continue to use it) with multiple clients. It uses Percona's Xtrabackup to take the backup (although it can be easily modified to use mysqldump instead).

First the script

-----------------------------------------------------------------------------------------------------------------

#!/bin/bash
SAVEIFS=$IFS
IFS=$""
day_of_week=`date +%a`
backup_dir=/mysql-backup/
logfile=/root/backup_log.txt
report=/tmp/report.txt
servername=slave1
email=bmurphy@paragon-cs.com
password=`cat /root/.ssh/.backup_password`

# run backup
echo ' ' > $report
echo 'The backup is now beginning:'












  [Read more...]
Interviewing tip..
+0 Vote Up -0Vote Down

I've been involved in a number of interviews over the last few weeks as a client has been loking for a MySQL DBA. When you are looking for position as a DBA in a large scale environement there are some very important things you have to know.

You absolutely must know a scripting language. In a smaller environment this often isn't necessary. You will live and die by this in a large environment. I asked every applicant one specific question..if you had to change a mysql server variable on a pool of 100 mysql servers how would you do this? It's easy when it's one,two or even a dozen servers. just log in, change the my.cnf and change it "on the fly" if you can. Restart mysql if you can't.

You going to do that to 100 servers? It will take all day and be prone to failures. Scripting is the key here. Even just bash shell scripting can be very powerful. In another

  [Read more...]
How To Sort Columns of MySQL Data on a Web Page With Perl
+0 Vote Up -0Vote Down

A friend of mine was building a web site so his customers could view his current inventory of transportation containers, and he asked me for help on how to sort the rows of information that appeared on his site. So, in this post, I will give you a quick example on how to sort columns on a web page.

First, let’s start with an inventory database that we will build in MySQL (http://mysql.com):

CREATE TABLE `inventory` (
`id` int(6) NOT NULL AUTO_INCREMENT,
`item_name` varchar(30) NOT NULL,
`item_SKU` varchar(20) NOT NULL,
`item_cost` decimal(4,2) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=100000 DEFAULT CHARSET=latin1

Next, here are some SQL statements to populate the MySQL database with some sample data.

use inventory;





  [Read more...]
Automatically Download MySQL Enterprise Monitor Graphs as PNG Files Using Perl
+0 Vote Up -0Vote Down

I was giving a presentation of the MySQL’s Enterprise Monitor (http://www.mysql.com/products/enterprise/monitor.html)* application to a client recently. I was demonstrating the “graphs” section of MEM, where you can monitor MySQL sessions, connections, replication latency and more with 60+ graphs. Usually, you view the graphs from within the MEM Enterprise Dashboard (via a web browser). But the client asked if there was a way to automatically download graphs. I wasn’t sure why he wanted to download the graphs (I didn’t ask), but I knew it wasn’t possible by using MEM alone. However, in the past I have written Perl scripts to automatically download files from web sites, so I thought I would see if it was possible with MEM.

 
*The MySQL Enterprise Monitor (MEM)
  [Read more...]
Using MySQL and Perl to Create, Edit and Delete Information Via a Web Page
+0 Vote Up -0Vote Down

A friend of mine was asking me for my recommendation of a good desktop database program to use to keep track of his inventory of cargo containers. I suggested to him that he should use MySQL and write a web page interface to do everything that he needed. He then reminded me that he is a lawyer by trade, and that he doesn’t have any computer programming experience. Then I remembered that he has almost zero computer skills. And his Texas Hold-Em skills are even worse, but I don’t mind taking his money. In his case, he should just use a notepad and a pencil. (As for the question – what is a lawyer doing with cargo containers? – that is a different story.)

If he did decide to broaden his horizons a bit, he could easily write his own software web application for creating and storing almost any

  [Read more...]
Checking on the Progress of Large DML Commands in MySQL Using Perl – Part Two
+0 Vote Up -0Vote Down

Part Two of Two: Checking on database activity when running a large DML (Data Manipulation Language) statement – such as INSERT, DELETE, UPDATE or SELECT.

Part Two: Monitoring the activity via Perl and SHOW ENGINE INNODB STATUS. (part of the InnoDB Monitors)

In part one, I showed you how to use a Perl script to insert a million rows of dummy data into a table. I needed a large database in order to test a Perl script that I would use to monitor the activity when running a large DML statement.

The original reason for creating both of these scripts was to find a quick way to see if a large

  [Read more...]
Checking on the Progress of Large DML Commands in MySQL Using Perl – Part One
+1 Vote Up -0Vote Down

Part One of Two: Checking on database activity when running a large DML (Data Manipulation Language) statement – such as INSERT, DELETE, UPDATE or SELECT.

Part One: Inserting a million rows into a database.

A friend of mine had asked a question – “Is there any way you can track how far you have advanced in a slow-moving ALTER or OPTIMIZE statement?”. A customer was performing some modifications on a database with tens of millions of rows, and they wanted to be able to see if the command was making any progress.

Since the customer was using the InnoDB storage engine, I thought of a way that you could check on the progress – but only given

  [Read more...]
Using MySQL, Perl and jQuery to Auto-Populate a Form Field on a Web Page
+0 Vote Up -0Vote Down

If you have ever built a form on a web page, you might have used a drop-down menu to display the choices available for a particular field. With a drop-down menu, you restrict the choices a user may select so that the user doesn’t enter invalid data (among other reasons). If a user misspells an entry, then a subsequent search for that value would not produce a found result.

A friend of mine who runs an online forum sent me an email about a problem he was having. He was trying to modify an existing registration web page using jQuery to auto-populate the state names, and then pass the state abbreviation back to his MySQL database. Believe it or not, he was actually having problems with people knowing their own state abbreviation. He had searched and found an example of what he wanted to do, but he couldn’t get

  [Read more...]
Steve Jobs, you will be missed. Greatly.
+0 Vote Up -0Vote Down

I remember my first computer. It was a TI-99/4A. I bought it back in 1982 (I think), and it cost around $300. The entire computer fit inside what looked like a really thick keyboard. It had a slot on the right for cartridges, and I had a cassette tape drive that I used for backing up the BASIC computer programs that I wrote. I thought that it was a great computer at the time, but I really didn’t have anything to compare it to. The games were the best part of the computer (really the only fun part – my BASIC skills were lacking). And even though the games were fairly lame by even 1982 standards, but they were still plenty of fun to play.

I also remember when I saw a Macintosh for the first time. I had followed Apple for some time, but I had never had the opportunity to actually see a Macintosh. I think it was in 1987 or 1988, when I was

  [Read more...]
Using MySQL to Import and Retrieve Blobs and Display as Image Files in HTML
+3 Vote Up -1Vote Down

I received a phone call from a friend of mine who had some questions about storing image files as blobs in MySQL. He wanted to design a web site that would allow someone to upload an image, and then that image would be stored in the MySQL database for viewing later. He stated that he was going to be storing thousands of images. I mentioned that it might be better to store the images on the file system, and then to just store the location of the image in the database. But, he still wanted to know how to do it and would decide which solution he would incorporate later.

I already had a Perl script that allowed me to upload files to a web site, as I would give out this URL to people that wanted to send me large files. And, I know that you can store images (and other files) in MySQL as a blob, but I wasn’t sure

  [Read more...]
Splitting a MySQL Dump File Into Smaller Files Via Perl
+1 Vote Up -1Vote Down

I was trolling the MySQL forums web site and noticed a post regarding someone who was trying to load a 50-gigabyte MySQL dump file. The author of the post had stated that the loading of the file had been running for days, and was wondering if there was a better way to import the file. They did not have access to anything else (i.e. – original database) but this file.

I have had to restore several databases in the past from a single large MySQL dump file – which led me to start backing up each database individually. These databases are for my own personal use and are not updated that often, so I don’t need to have point-in-time recovery – and so a MySQL dump works just fine. If I had a production system, I would invest in the MySQL Enterprise Backup and the MySQL

  [Read more...]
Perl TCP Listener for Detecting Available Ports for MySQL Enterprise Monitor
+2 Vote Up -0Vote Down

I recently visited a client for the purpose of installing and demonstrating MySQL Enterprise Monitor (http://www.mysql.com/products/enterprise/monitor.html).

If you are unfamiliar with the MySQL Enterprise Monitor – from the MySQL web site: The MySQL Enterprise Monitor continuously monitors your MySQL servers and alerts you to potential problems before they impact your system. Its like having a “Virtual DBA Assistant” at your side to recommend best practices to eliminate security vulnerabilities, improve replication, optimize performance and more. As a result, the productivity of your developers, DBAs and System Administrators is improved significantly.

The MySQL Enterprise Monitor is a distributed web application that is deployed within the safety of your firewall. It is comprised of a

  [Read more...]
Convert .csv File to MySQL Database via Perl
+1 Vote Up -0Vote Down

Have you ever had a spreadsheet file or a large .csv file that you wanted to manipulate, but you want more power than a spreadsheet program could offer?

Before I started using MySQL, I would usually throw the .csv file into a desktop database program, like FileMaker. FileMaker would allow you to import the .csv file and it would automatically create the column headers for you. Recently, I was given a spreadsheet with 27,000 rows in it. I still use FileMaker for some databases, but I wanted the power of MySQL to manipulate the information contained in this file. So, I could have easily just typed out the database column names manually into a MySQL “create table” statement, guessed at the types and sizes of the columns and then imported the .csv file. Instead, I decided to write a Perl script to do the dirty work for me. Plus, this

  [Read more...]
Showing entries 1 to 30 of 90 Next 30 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.