In this post I’ll expand on the subject of my MySQL pre-FOSDEM talk: what dbadmins need to know and do, when upgrading from MySQL 5.7 to 8.0.
I’ve already published two posts on two specific issues; in this article, I’ll give the complete picture.
As usual, I’ll use this post to introduce tooling concepts that may be useful in generic system administration.
The presentation code is hosted on a GitHub repository (including the …
[Read more]When it comes to configuring MySQL, a fundamental step is to find out which configuration files the MySQL server reads.
The operation itself is simple, however, if we want to script the operation, using text processing in a sharp way, it’s not immediate what the best solution is.
In this post I’ll explore the process of looking for a satisfying solution, going through grep, perl, and awk.
Contents:
[Read more]
We’re in the process of upgrading our MySQL databases from v5.7
to v8.0; since one of the differences in v8.0 is that the default
encoding changed from utf8 to utf8mb4,
and we had the conversion in plan anyway, we anticipated it and
performed it as preliminary step for the upgrade.
This post describes in depth the overall experience, including tooling and pitfalls, and related subjects.
Contents:
[Read more]October 9, 2014 By Severalnines
Installing, configuring, deploying databases and performing repetitive administrative tasks are all part of a DBA’s or sysadmin’s job. This can get pretty repetitive and overwhelming if you are part of a centralized IT team, running multiple databases for your organization’s different departments, or a managed hosting provider responsible for setting up and operating databases for external clients. One way to get out of this ‘manual, repetitive task’ business is through a Database as a Service (DBaaS).
DBaaS is a way of delivering database functionality as a service to one or more consumers. A DBaaS platform would provide automated procedures for database deployment, monitoring, backups, recovery/repair, scaling, security/multi-tenancy, etc. This type of automation is especially useful where agility is needed, e.g. for systems that require elasticity by scaling out or scaling back at short …
[Read more]I recently worked on a Rails 3.2 project that used the sweet PLupload JavaScript/Flash upload tool to upload files to the web app. To make it easier for users to upload large and/or remote files to the app, we also wanted to let them upload via SFTP. The catch was, our users didn't have SFTP accounts on our server and we didn't want to get into the business of creating and managing SFTP accounts. Enter: ProFTPD and virtual users.
ProFTPD's virtual users concept allows you to point ProFTPD at a SQL database for your user and group authentication. This means SFTP logins don't need actual system logins (although you can mix and match if you want). Naturally, this is perfect for dynamically creating and destroying SFTP accounts. …
[Read more]Converting from uw-mailboxes (mbx) to Unix format (dovecot)
It took me by surprise how the mailbox formats had changed, when I switched to Dovecot which is the best IMAP/POP3 mail program, in my opinion. It handles large (enormous) mailboxes with many (hundreds of) users. Caching makes things go fast again.
Here is a simple bash utility, to take all users, and convert all
mailboxes & folders.
cd /home
for u in *
do
if [ -d /home/$u/Mail ]; then
echo "User: $u"
cd /home/$u/Mail
On Linux, you have a plethora of File Systems available.
Recently I was testing databases (MySQL, MariaDB) especially with ThreadPool against a Violin Flash Memory Array, and I forgot what file system the LUN was mounted (and formatted) as.
There are two ways to find out:
A set of internet tools, is available at otala.net/tools.shtml
More to come, as (if/when) I get time -- suggestions welcome
I found Dennis the Menace, he now has a job as system administrator for a hosting company. Scenario: client has a problem with a server becoming unavailable (cause unknown) and has it restarted. MySQL had some page corruption in the InnoDB tablespace.
The hosting provider, being really helpful, goes in as root and first deletes ib_logfile* then ib* in /var/lib/mysql. He later says “I am sorry if I deleted it. I thought I deleted the log only. Sorry again.” Now this may appear nice, but people who know what they’re doing with MySQL will realise that deleting the iblogfiles actually destroys data also. MySQL of course screams loudly that while it has FRM files it can’t find the tables. No kidding!
Then, while he’s been told to not touch anything any more, and I’m trying to see if I can recover the deleted files on ext3 filesystem (yes there are tools for that), he goes in again and puts an ibdata1 file back. No, …
[Read more]
CFEngine is
both the oldest and the newest of the popular tools for
automating site administration. Mark Burgess invented it as a
free software project in 1993, and years later, as deployments in
the field outgrew its original design he gave it a complete
rethink and developed the powerful concept of promise
theory to make it modular and maintainable. In this guise as
version 3, CFEngine stands along with two other pieces of free
software, Puppet and Chef, as key parts of enterprise computing.
Along the way, Burgess also started a commercial venture,
CFEngine AS, that maintains both the open source and proprietary
versions of CFEngine.
Diego Zamboni has recently taken the position of Senior Security Advisor at CFEngine AS and is writing a book for O'Reilly on CFEngine 3. I talked to him this week about the recent new …
[Read more]