Showing entries 1 to 7
Displaying posts with tag: load (reset)
Tech Messages | 2012-05-11

A special extended edition of Tech Messages for 2011-09-21 through 2012-05-11:

Improving MySQL Insert thoughput

There are three simple practices that can improve general INSERT throughput. Each requires consideration on how the data is collected and what is acceptable data loss in a disaster.

General inserting of rows can be performed as single INSERT’s for example.

INSERT INTO table (col1, col2, col3) VALUES (?, ?, ?);
INSERT INTO table (col1, col2, col3) VALUES (?, ?, ?);
INSERT INTO table (col1, col2, col3) VALUES (?, ?, ?);

While this works, there are two scalability limitations. First is the network overhead of the back and forth of each SQL statement, the second is the synchronous nature, that is your code can not continue until your INSERT is successfully completed.

The first improvement is to use MySQL’s multi values capability with INSERT. That is you can insert multiple rows with a single INSERT statement. For example:

INSERT INTO table (col1, col2, col3) VALUES (?, ?, ?),  (?, ?, ?), (?, ?, ?);
[Read more]
mg_hot_replace_table.pl

Do you have MyISAM tables you reload with new data?

Do your queries, using that table, get blocked because the table is locked?

Do the waiting queries create idle connections slowing down the table load?

Do you wish you could just replace the table?

Years ago I was told you can replace CSV tables by simply replacing the CSV file. I figured this would also be true of a MyISAM file and it is. I use this perl script to replace MyISAM tables forcast and current observation weather data. The processing and tables are created on another computer. Weather forecasting is CPU and database expensive. I then copy (rsync) the files to the production system and run this script.

#!/usr/bin/perl
################################################################################
################################################################################
# mg_hot_replace_table.pl - Hot Replace a MySQL table.
#
# 2010-05-01 …
[Read more]
Linux Open Administration Days 2010

So about 4 monts ago there was the crazy idea to start a new FOSS event in Belgium targeted at sysadmins.

What started out as an event for local people to meet local people with some local speakers actually ended up being a small local event with some top international speakers on onfiguration mananagement and system administration mixed with a bunch of good local ones !

I had the honour to open the conference with an extremely short version of the Devops talk I gave earlier last year.. extremely short as I knew that over the course of the weekend the topic would reoccur a lot.

We had the first european talk on Chef, by Joshua Timberman, and we had Puppet talks amongst by Dan Bode from Puppetlabs and CFengine talks , devops was a frequently dropped word,

We had a book raffle where …

[Read more]
Follow-up To Loading CSS And JS Conditionally

First of all, I'd like to thank everyone who read and gave their 2 cents about the [WordPress Plugin Development] How To Include CSS and JavaScript Conditionally And Only When Needed By The Posts post. The article was well received and will hopefully spark some optimizations around loading styles and scripts.

Here are some discussions and mentions around the web:

[Read more]
Solaris Cluster 3.2 11/09 is now available

Solaris Cluster is a multi-system, multi-site high availability and disaster recovery solution that manages the availability of applications services and data across local, regional and geographically dispersed data centers. The Solaris Cluster environment extends the Solaris Operating System into a cluster operating system.

If you are following the Solaris Cluster product and features, you would have noticed extreme innovation by Sun in the high availability and disaster recovery space since the time we released our first HA product many years ago. For well over a decade, Solaris Cluster has been a market leader for providing business continuity and disaster recovery solutions to all mission critical business applications, spanning all the major industry segments.

Continuing with our tradition of innovation, we are pleased to announce another release - "Solaris Cluster 3.2 11/09" - an update to the Solaris Cluster 3.2 …

[Read more]
Solaris Cluster 3.2 11/09 is now available

Solaris Cluster is a multi-system, multi-site high availability and disaster recovery solution that manages the availability of applications services and data across local, regional and geographically dispersed data centers. The Solaris Cluster environment extends the Solaris Operating System into a cluster operating system.

If you are following the Solaris Cluster product and features, you would have noticed extreme innovation by Sun in the high availability and disaster recovery space since the time we released our first HA product many years ago. For well over a decade, Solaris Cluster has been a market leader for providing business continuity and disaster recovery solutions to all mission critical business applications, spanning all the major industry segments.

Continuing with our tradition of innovation, we are pleased to announce another release - "Solaris Cluster 3.2 11/09" - an update to the Solaris Cluster 3.2 …

[Read more]
Showing entries 1 to 7