Showing entries 1 to 10 of 109
10 Older Entries »
Displaying posts with tag: Scripting (reset)
A practical explanation: problems during unicode collation conversion

Introduction

Recently I have been involved in an effort to convert MySQL databases from a utf8 character set to utf8mb4. As a part of this effort, my team evaluated which collations would be best for facilitating a broad range of multi-lingual support.

There have been many recent posts in the MySQL community about better unicode collation support in MySQL 8 such as from the MySQL Server Team’s blog at Oracle, who have also done a good job of showing us how newer collations based on UTF8 9.0.0 will properly group and sort characters according to their case and inheritance. As the title of the latter post suggests, the “devil is” indeed “in the details”.

There is also the matter of the …

[Read more]
Easy-to-use Perl scripts to backup your MySQL database with mysqldump and FTP the files to a remote server

Most users of MySQL utilize the mysqldump utility to backup their database. While mysqldump is handy and easy-to-use (and free), if you have data which is important to your business, then you should take a look at the MySQL Enterprise Edition – and use MySQL Enterprise Backup instead. The MySQL Enterprise Backup allows you to backup your database without the table locking you get with mysqldump. And, it is extremely fast – especially when you have to restore your database. Here is a sample speed comparison between MySQL Enterprise Backup and …

[Read more]
Binary Log Growth Handling in MySQL

So today we look at an aspect of capacity management or planning and how it relates to MySQL and its Logging.  For this blog I’m primarily going to look at MySQL’s binary logs. They have a tendency to grow, and will continue to if not held in check.  Happily, they tend to be easy to… Read More »

MySQL multi-instance Group Replication on systemd

In this blog post, I’d like to take a look at a few different things such as MySQL Group Replication, multi-instance MySQL setups on systemd and shell scripting the whole mess to make it easy to build, and easy to rebuild.  To be honest, it took a little help from Shinguz’s blog to get the… Read More »

Command line QPS (Queries Per Second) Quick and Dirty

Ever wanted to just look at QPS in real time while logged into your server?

Well here’s a little command line hackery to do it quick and dirty.

[user@yourserver ~] $ LASTVAL=0; while true; do CURVAL=`mysql --batch -N -e "show status like 'Quer%';" | awk '{print $2}'`; QPS=`expr $CURVAL - $LASTVAL`; if [ $LASTVAL -ne 0 ]; then echo "$CURVAL $QPS"; fi; LASTVAL=$CURVAL; sleep 1; done

The output looks like this:

65549603430 2439
65549605421 1991
65549606912 1491
65549611219 4307
65549614186 2967
65549618048 3862
65549620853 2805

The first column is just the Query counter value. The second column is the QPS.

This script requires a .my.cnf to exist in your home directory (or that you do something nastily insecure and supply -u user -ppassword to the mysql command in the example above).


[Read more]
Fetch CSV of MySQL table size vs .ibd container size

This only works if you’re using innodb_file_per_table.

Purpose: import this csv quickly into google sheets (or other spreadsheet) and compare MySQL’s internal data size to the container size on disk to determine tables needing to be optimized (or “null altered”) to reclaim disk space and maybe increase performance due to defragmentation. Rule of thumb is probably something like >=10% difference may warrant action.

I wrote this loop as a one-liner dynamically / ad-hoc on the command line a couple weeks ago but made it into a configurable, yet quick-and-dirty shell script, below.

Add -u and -p arguments to MySQL CLI command if you need to, or just place a .my.cnf in your home directory and use the script as-is.

#!/bin/bash

DATADIR="/path/to/datadir" # ex: /var/lib/mysql
SCHEMANAME="yourschema"

for x in `mysql --batch -n -e "select concat(concat(table_name,'.ibd'),',',(data_length + index_length)) as …
[Read more]
MySQL Failover, Enhanced MySQL Utilities

This blog is a 2nd part of a multi-part series on areas of  failover for MySQL.  The first installment looked at design considerations, giving us a “thinking” perspective on what we might want to adopt.  Later I will take a look at more of a business and operational way of thinking through these details.  In… Read More »

MySQL Failover Design Considerations

This will be a multi-part series covering various areas of failover for MySQL.  This first installment will primarily look at some design considerations, which you can then apply to your own environment in your own way.  The concepts presented here are merely suggestions and not out-right “how-to”.   Every company has specific technologies or skill-sets in… Read More »

Use MySQL to store data from Amazon’s API via Perl scripts

I really like Amazon.com and I have been a Prime member for several years. Along with millions of different items for sale, Amazon has an affiliate program, where you can earn money advertising products on your web site. When a visitor to your site clicks on a link and orders a product from Amazon, you earn a small commission on the total sale. As an affiliate, you have access to Amazon’s Product Advertising API for obtaining product information. But, you can use this information for many other purposes.

The Amazon API is like most other API’s, and their API web site provides you with code examples and explains how it all works. I am going to show you how a Perl program which you can use to access the API data and store it in a …

[Read more]
Add RSS feeds to your Twitter stream using MySQL and Perl

Adding good content to Twitter can be a pain. I can’t do it during working hours, and I don’t have much time at night. But, the more content you have, the more followers you can gain, and the more your original tweets can be seen (hopefully). I have written several posts about using the latest Perl-Twitter API – Net::Twitter::Lite::WithAPIv1_1, so you might want to check these out as well.

Use MySQL and Perl to automatically find, follow and unfollow twitter users

Using Perl to retrieve direct messages from Twitter, insert messages into a MySQL database and then …

[Read more]
Showing entries 1 to 10 of 109
10 Older Entries »