I actively maintain virtual test instances for my writing and teaching on Windows 7, Red Hat Enterprise Linux, Fedora, and Solaris. Perl on Windows is always interesting. I use ActivePerl on Windows 7 rather than Strawberry Perl, and it was interesting to see this note after I upgraded to the most current Community Edition of ActivePerl (5.16.3).

I thought it might be kind to post ActiveState’s Release Note because you should really read it before you try to install ActivePerl on Windows 7/8. Just make sure you’ve removed any earlier version of ActivePerl before trying the install. That’s what I did, and surprise, there weren’t any problems.

While the installation doesn’t tell you that you need to restart the Windows 7 operating system, you do. If you don’t restart Windows 7 after the ActivePerl install, you’ll get the

This month is a special month. It’s not because of President’s Day or even the exciting day where we revel in groundhogs. No, this month is special because the free book give-away is happening again. This is where you, the reader, gets to win something free for doing nothing more than posting a comment saying that you want a copy of my recently published book – The InnoDB Quick Reference Guide from Packt Publishing. The book is a great reference for DBAs, PHP, Python, or Perl programmers that integrate with MySQL and want to learn more about the InnoDB database engine.

So, all you have to do is post a comment here saying that you want a copy and write out a single (or more) sentence about how you use InnoDB in your

I’m pleased to announce that my first book, the InnoDB Quick Reference Guide, is now available from Packt Publishing and you can download it by clicking here. It covers the most common topics of InnoDB usage in the enterprise, including: general overview of its use and benefits, detailed explanation of seventeen static variables and seven dynamic variables, load testing methodology, maintenance and monitoring, as well as troubleshooting and useful analytics for the engine. The current version of MySQL ships with InnoDB as the default table engine, so whether you program your MySQL enabled applications with PHP, Python, Perl or otherwise, you’ll likely benefit from this

Before I upgrade MySQL to the latest and greatest version, one of the first things that I do is export the user and grant information. In the past, I would keep all of my user information (user name, password, grants) in a text file, with the SQL for each user/grant ready to be executed on the upgraded server. I did use my own form of “mental encryption” for my passwords, so the passwords weren’t in plain English. But then I would have to decode my passwords each time before I executed the SQL statements.

When I upgrade, I usually like to dump all of the data and import it into the new version, so I have a fresh copy of the database. The MySQL server that I have is for my personal use and the data size is relatively small, so for my case it doesn’t take long to import the data.

But there were times when I

MySQL MHA by Oracle ACE Director Yoshinori Matsunobu is an excellent open source tool to help in providing HA with native MySQL replication. The installation however is dependent on some Perl packages and to the untrained eye this may be an issue if you are using Percona Server as your choice of MySQL implementation.

The MHA Node page requires the perl-DBD-MySQL package to be installed. The installation on RedHat/CentOS/Oracle Linux look like this:

$ sudo yum install perl-DBD-MySQL
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.anl.gov
* extras: mirror.anl.gov
* updates: mirror.anl.gov
Setting up Install Process
Resolving

I was digging through some old project code and found this script. Sometimes one finds oneself in an odd situation and needs to convert regular SQL, say from a MySQL database dump, into SQLite3 format. There’s not too much else to say, but here is a script that helps with the process. It can likely be improved but this handles the items that came up during conversion on initial runs.

#!/bin/sh
####
# NAME: convert-mysql-to-sqlite3.sh
# AUTHOR: Matt Reid
# DATE: 2011-03-22
# LICENSE: BSD
####
if [ "x$1" == "x" ]; then
   echo "Usage: $0 "
   exit 
fi 
cat $1 |
grep -v ' KEY "' |   
grep -v ' UNIQUE KEY "' |
grep -v ' PRIMARY KEY ' |
sed '/^SET/d' |          
sed 's/ unsigned / /g' | 
sed 's/ auto_increment/ primary key autoincrement/g' |
sed 's/ smallint([0-9]*) / integer /g' | 
sed 's/ tinyint([0-9]*) / integer /g' |  
sed 's/ int([0-9]*) / integer /g' |      
sed 's/

Using a lowercase hostname is typical but I got sloppy on a Windows 7 installation, after all Windows is case insensitive, and I used a mixed case hostname. It raised an interesting error when installing Oracle Database 11g Release 2.

Failed to allocate port(s) in the specified range(s) for the following process(es): JMS
[5540-5559], RMI [5520-5539], Database Control [5500-5519], EM Agent [3938] | [1830-1849]

Refer to the log file at C:\app\McLaughlinM\cfgtoollogs\dbca\orcl\emConfig.log for more details.

You can retry configuring this database with Enterprise Manager
  [Read more...]

I feel a sense of pride when I think that I was involved in the development and maintenance of what was probably the first piece of software accepted into Debian which then had and still has direct up-stream support from Microsoft. The world is a better place for having Microsoft in it. The first operating system I ever ran on an 08086-based CPU was MS-DOS 2.x. I remember how thrilled I was when we got to see how my friend’s 80286 system ran BBS software that would cause a modem to dial a local system and display the application as if it were running on a local machine. Totally sweet.

When we were living at 6162 NE Middle in the nine-eight 292, we got an 80386 which ran Doom. Yeah, the original one, not the fancy new one with the double barrel shotgun, but it would probably run that one, too.

The brief outage was due to a scheduled move of the servers to a separate rack and subnet dedicated to our work with the Center for Information Assurance & Cybersecurity (ciac) at the University of Washington Bothell (uwb), and a11y.com

I am currently exercising the new (to us) equipment and hope to winnow the less than awesome equipment over the next quarter. I spent the last six months finding the best in breed of the surplussed DL385 and DL380 chassis we (work) were going to have recycled. The team and I were able to find enough equipment to bring up one of each with eight and six gigs of memory, respectively. These will make excellent hypervisors for provisioning embedded instances of Slackware, Fedora, RHEL, CentOS, Debian, FreeBSD, OpenSolaris, OpenIndiana, FreeDOS, etc.

When I initially configured this xen paravirt environment, I failed to plan for integration with libvirt, so I am

It’s been released. Use this with NIST::NVD 1.00.00 and you will be able to perform immediate look-ups of CVE and CWE data given a CPE URN. For instance:

cjac@foxtrot:/usr/src/git/f5/NIST-NVD-Store-SQLite3$ perl Makefile.PL ; make ; make test ; cjac@foxtrot:/usr/src/git/f5/NIST-NVD-Store-SQLite3$ perl -MNIST::NVD::Query -MData::Dumper -e '
$q = NIST::NVD::Query->new(store    => q{SQLite3},database => q{t/data/nvdcve-2.0.db});
$cve_list = $q->cve_for_cpe( cpe => q{cpe:/a:microsoft:ie:7.0.5730.11} );
print Data::Dumper::Dumper { cve_list => $cve_list, first_cvss => $q->cve( cve_id => $cve_list->[0] )->{q{vuln:cvss}} }
'
$VAR1 = {
          'cve_list' => [
                          'CVE-2002-2435',
                          'CVE-2010-5071'
                        ],
          'first_cvss' => {

I’m leaving myself some room for bug fixes. It works for us in house. I would love to help others to give it a try. especially those who could benefit from making nearly immediately answered queries to the NIST’s NVD database.

The code in this release cannot by itself track the feed from the feds in real time. The nvd entry loader needs a little bit of love in the area of record merging before this starts working. It’s on my TODO list.

I’m sorry for the outage of git.colliertech.org. I’ll get that back up here shortly. In the meantime, feel free to grab it from this location while the CPAN indexes and processes my submission.

http://www.colliertech.org/federal/NIST/NIST-NVD-1.00.00.tar.bz2

don’t forget to check the cryptographic signature:

I've been involved in a number of interviews over the last few weeks as a client has been loking for a MySQL DBA. When you are looking for position as a DBA in a large scale environement there are some very important things you have to know.

You absolutely must know a scripting language. In a smaller environment this often isn't necessary. You will live and die by this in a large environment. I asked every applicant one specific question..if you had to change a mysql server variable on a pool of 100 mysql servers how would you do this? It's easy when it's one,two or even a dozen servers. just log in, change the my.cnf and change it "on the fly" if you can. Restart mysql if you can't.

You going to do that to 100 servers? It will take all day and be prone to failures. Scripting is the key here. Even just bash shell scripting can be very powerful. In another

I’m in the process of getting the tests passing for the 0.03 release of NIST::NVD::Store::SQLite3 wherein our hero imports the CWE data and cross-indexes it with CVEs and CPEs.

Follow along and suggest some patches. I’m developing on Debian Wheezy, but I would very much like input from devs on other platforms.

http://git.colliertech.org/?p=NIST-NVD-Store-SQLite3.git;a=summary

cjac@foxtrot:/tmp$ time git clone http://git.colliertech.org/git/NIST-NVD-Store-SQLite3.git
Cloning into 'NIST-NVD-Store-SQLite3'...

real	0m32.757s
user	0m0.200s
sys	0m0.088s
cjac@foxtrot:/tmp$ ls NIST-NVD-Store-SQLite3/t/data/
cwec_v2.1.xml  nvdcve-2.0-test.xml

Publish your patches and I’ll fetch them, or you can submit them in udiff format and I’ll review/apply. Thanks for playing

I published an SQLite3 storage back-end to NIST::NVD on the CPAN. It’s pretty quick. About as fast as the DB_File one, but without the down side of being tied to DB_File. It shouldn’t be too difficult to re-factor this code to any DBI-based database. MariaDB anyone?

I know it works on Debian. The nightly CPAN test results should come back shortly and I’ll find out how well it works on other platforms.

Pluggable M3 (Monitis Monitor Manager) Framework

Who needs an introduction about M3? – Perhaps no one!
After gaining some reputation with M3, providing extra-easy integration of any monitor into Monitis it was time to take it to the next level.

Generally speaking, the work flow of M3 was described in detail in this article.

After some thought and design, we’ve decided it’d be best if M3 was pluggable. Pluggable in

For a work project, I wrote a library in perl that can be used to query the NVD feed that NIST publishes here:

http://nvd.nist.gov/download.cfm

Here’s a snippit from the perldoc:

use NIST::NVD::Query;
 
# use convert_nvdcve to generate these files from the XML dumps at
# http://nvd.nist.gov/download.cfm
 
my( $path_to_db, $path_to_idx_cpe ) = @ARGV;
 
my $q = NIST::NVD::Query->new( database => $path_to_db,
                               idx_cpe  => $path_to_idx_cpe,
                              );
 
# Given a Common Platform Enumeration urn, returns a list of known
# CVE IDs
 
my $cve_id_list = $q->cve_for_cpe( cpe => 'cpe:/a:zaal:tgt:1.0.6' );
 
my @entry;
 
foreach my $cve_id ( @$cve_id_list ){
 
  # Given a CVE ID, returns a CVE

A friend of mine was building a web site so his customers could view his current inventory of transportation containers, and he asked me for help on how to sort the rows of information that appeared on his site. So, in this post, I will give you a quick example on how to sort columns on a web page.

First, let’s start with an inventory database that we will build in MySQL (http://mysql.com):

Next, here are some SQL statements to populate the MySQL database with some sample data.

I was giving a presentation of the MySQL’s Enterprise Monitor (http://www.mysql.com/products/enterprise/monitor.html)* application to a client recently. I was demonstrating the “graphs” section of MEM, where you can monitor MySQL sessions, connections, replication latency and more with 60+ graphs. Usually, you view the graphs from within the MEM Enterprise Dashboard (via a web browser). But the client asked if there was a way to automatically download graphs. I wasn’t sure why he wanted to download the graphs (I didn’t ask), but I knew it wasn’t possible by using MEM alone. However, in the past I have written Perl scripts to automatically download files from web sites, so I thought I would see if it was possible with MEM.

A friend of mine was asking me for my recommendation of a good desktop database program to use to keep track of his inventory of cargo containers. I suggested to him that he should use MySQL and write a web page interface to do everything that he needed. He then reminded me that he is a lawyer by trade, and that he doesn’t have any computer programming experience. Then I remembered that he has almost zero computer skills. And his Texas Hold-Em skills are even worse, but I don’t mind taking his money. In his case, he should just use a notepad and a pencil. (As for the question – what is a lawyer doing with cargo containers? – that is a different story.)

If he did decide to broaden his horizons a bit, he could easily write his own software web application for creating and storing almost any

Part Two of Two: Checking on database activity when running a large DML (Data Manipulation Language) statement – such as INSERT, DELETE, UPDATE or SELECT.

Part Two: Monitoring the activity via Perl and SHOW ENGINE INNODB STATUS. (part of the InnoDB Monitors)

In part one, I showed you how to use a Perl script to insert a million rows of dummy data into a table. I needed a large database in order to test a Perl script that I would use to monitor the activity when running a large DML statement.

The original reason for creating both of these scripts was to find a quick way to see if a large

Part One of Two: Checking on database activity when running a large DML (Data Manipulation Language) statement – such as INSERT, DELETE, UPDATE or SELECT.

Part One: Inserting a million rows into a database.

A friend of mine had asked a question – “Is there any way you can track how far you have advanced in a slow-moving ALTER or OPTIMIZE statement?”. A customer was performing some modifications on a database with tens of millions of rows, and they wanted to be able to see if the command was making any progress.

Since the customer was using the InnoDB storage engine, I thought of a way that you could check on the progress – but only given

If you have ever built a form on a web page, you might have used a drop-down menu to display the choices available for a particular field. With a drop-down menu, you restrict the choices a user may select so that the user doesn’t enter invalid data (among other reasons). If a user misspells an entry, then a subsequent search for that value would not produce a found result.

A friend of mine who runs an online forum sent me an email about a problem he was having. He was trying to modify an existing registration web page using jQuery to auto-populate the state names, and then pass the state abbreviation back to his MySQL database. Believe it or not, he was actually having problems with people knowing their own state abbreviation. He had searched and found an example of what he wanted to do, but he couldn’t get

This article is just a how-to for setting up Perl on Windows in order to use the perl scripts provided with MySQL, such as mysqldumpslow.pl or mysql_explain_log.pl.

Now, you might say there is a section on this topic in the MySQL manual, to which I’d agree. But, this was for ActiveState Perl 5.6. The latest ActiveState Perl is 5.12, so thoese instructions are a bit out-of-date. Also, there are some helpful “User Comments” on that same page, but again, those are slightly out-of-date in the ActiveState 5.12. So, I thought I’d just post the steps I took in order to set this up.

First, download and install ActivePerl. Click the “Download ActivePerl 5.12.4 for Windows” button (choose either 32-bit or 64-bit). You’ll be

I received a phone call from a friend of mine who had some questions about storing image files as blobs in MySQL. He wanted to design a web site that would allow someone to upload an image, and then that image would be stored in the MySQL database for viewing later. He stated that he was going to be storing thousands of images. I mentioned that it might be better to store the images on the file system, and then to just store the location of the image in the database. But, he still wanted to know how to do it and would decide which solution he would incorporate later.

I already had a Perl script that allowed me to upload files to a web site, as I would give out this URL to people that wanted to send me large files. And, I know that you can store images (and other files) in MySQL as a blob, but I wasn’t sure