Showing entries 1 to 10 of 55
10 Older Entries »
Displaying posts with tag: bash (reset)
Handy JSON to MySQL Loading Script

JSON in Flat File to MySQL DatabaseSo how do you load that JSON data file into MySQL. Recently I had this question presented to me and I thought I would share a handy script I use to do such work. For this example I will use the US Zip (postal) codes from JSONAR. Download and unzip the file. The data file is named zips.json and it can not be bread directly into MySQL using the SOURCE command. It needs to have the information wrapped in a more palatable fashion.

head zips.json 
{ "city" : "AGAWAM", "loc" : [ -72.622739, 42.070206 ], "pop" : 15338, "state" : "MA", "_id" : "01001" }
{ "city" : "CUSHMAN", "loc" : [ -72.51564999999999, 42.377017 ], "pop" : 36963, "state" : "MA", "_id" : "01002" }
{ "city" : "BARRE", "loc" : [ -72.10835400000001, 42.409698 ], "pop" : 4546, "state" : "MA", "_id" : "01005" }
{ "city" : "BELCHERTOWN", "loc" : [ -72.41095300000001, …
[Read more]
Encrypt your –defaults-file

Encrypt your credentials using GPG

This blog post will look how to use encryption to secure your database credentials.

In the recent blog post Use MySQL Shell Securely from Bash, there are some good examples of how you might avoid using a ~/.my.cnf – but you still need to put that password down on disk in the script. MySQL 5.6.6 and later introduced the  –login-path option, which is a handy way to store per-connection entries and keep the credentials in an encrypted format. This is a great improvement, but as shown in Get MySQL Passwords in Plain Text from .mylogin.cnf, …

[Read more]
Use MySQL Shell Securely from Bash

This blog post discusses how to use MySQL shell securely from Bash.

The Bourne shell is everywhere. It is part of the most basic Linux install. You will find it on the biggest SPARC machines down to a Raspberry Pi. It is nice to know it will always be there. Unlike other, more complex scripting environments such as Perl and Python, it doesn’t require any additional dependencies to be installed.

Anyone that has automated a MySQL task using a Bourne shell such as Bash will be familiar with the following message:

Warning: Using a password on the command line interface can be insecure.

This semi-ominous warning describes a security flaw in passing credentials on a process command line. Any unprivileged user on the system can use a …

[Read more]
Log Buffer #444: A Carnival of the Vanities for DBAs

This Log Buffer Edition covers some blog posts of Oracle, SQL Server and MySQL from this past week.


  • Oracle Utilities Application Framework V4. includes a new help engine and changes to the organization of help.
  • Very simple oracle package for HTTPS and HTTP.
  • Tim spotted a problem with the PDB Logging Clause.
  • How to Pass Arguments to OS Shell Script from Oracle Database.

SQL Server:

  • How efficient is your covered …
[Read more]
Simplifying Docker Interactions with BASH Aliases

Docker has been consuming my life in the last few weeks. I have half a dozen projects in progress that use containers in some fashion, including my Visualizing MySQL’s Performance Schema project.

Since I prefer to work from a Mac laptop, I have to utilize a Linux Virtual Machine (VM) which runs the Docker daemon. Luckily, Docker Machine makes this a very simple process.

However, interacting both with Docker and Docker Machine does introduce some additional commands that I would rather simplify for the repeatable use-cases I’ve come across. With BASH aliases, this is not a problem.

Is My Docker Environment Setup?

When …

[Read more]
Bash Arrays & Oracle

Last week, I wrote about how to use bash arrays and the MySQL database to create unit and integration test scripts. While the MySQL example was nice for some users, there were some others who wanted me to show how to write bash shell scripts for Oracle unit and integration testing. That’s what this blog post does.

If you don’t know much about bash shell, you should start with the prior post to learn about bash arrays, if-statements, and for-loops. In this blog post I only cover how to implement a bash shell script that runs SQL scripts in silent mode and then queries the database in silent mode and writes the output to an external file.

I’ve copied the basic ERD for the example because of a request from a reader. In their opinion it makes cross referencing the …

[Read more]
Bash Arrays & MySQL

Student questions are always interesting! They get me to think and to write. The question this time is: “How do I write a Bash Shell script to process multiple MySQL script files?” This post builds the following model (courtesy of MySQL Workbench) by using a bash shell script and MySQL script files, but there’s a disclaimer on this post. It shows both insecure and secure approaches and you should avoid the insecure ones.

It seems a quick refresher on how to use arrays in bash shell may be helpful. While it’s essential in a Linux environment, it’s seems not everyone masters the bash shell.

Especially, since I checked my …

[Read more]
MySQL, Percona, MariaDB long running processes clean up one liner

There are tools like pt-kill from the percona tool kit that may print/kill the long running transactions at MariaDB, MySQL or at Percona data instances, but a lot of backup scripts are just some simple bash lines.
So checking for long running transactions before the backup to be executed seems to be a step that is missed a lot.

Here is one line that might be just added in every bash script before the backup to be executed
Variant 1. Just log all the processlist entries and calculate which ones were running longer than TIMELIMIT:

$ export TIMELIMIT=70 && echo "$(date) : check for long runnig queries start:" >> /tmp/ && mysql -BN -e 'show processlist;' | tee -a /tmp/ | awk -vlongtime=${TIMELIMIT} '($6>longtime){print "kill "$1";"}' | tee -a /tmp/

Variant 2: Log all the processlist, calculate the calculate which processes …

[Read more]
SQL Developer – Fedora

This is the continuation of my efforts to stage an awesome Fedora developer’s instance. It shows you how to install Java 1.8 software development kit, which is nice to have. Though you can’t use Java 1.8 officially with Oracle SQL Developer 4.0.3 it is required for Oracle SQL Developer 4.1. Fortunately, the Oracle Product Manager, Jeff Smith has advised us that you can use Java 1.8 JDK with Oracle SQL Developer 4.0.3, and he’s written a comment to the blog post that it runs better with the Java 1.8 SDK.

After you install Oracle SQL Developer 4.0.3 or Oracle SQL Developer 4.1, you can watch Jeff Smith’s YouTube Video on SQL Developer 3.1 to learn how to use the basics of SQL Developer. I couldn’t find an updated version of the video for SQL Developer 4 but I didn’t try too hard.

You …

[Read more]
Add zsh to Fedora

One of my students requested an option to the bash shell. It was interesting to hear that he wanted me to instal the zsh in my Fedora image. There’s only one book that I’m aware of that’s been published on the Z Shell, and it is From Bash to Z Shell.

This post shows how to add the zsh to my Fedora image because I already release a new one for the term without the zsh shell. You use the yum utility as the root user to install the zsh library:

yum …
[Read more]
Showing entries 1 to 10 of 55
10 Older Entries »