Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 66 Next 30 Older Entries

Displaying posts with tag: Sysadmin (reset)

SFTP virtual users with ProFTPD and Rails: Part 1
+0 Vote Up -1Vote Down

I recently worked on a Rails 3.2 project that used the sweet PLupload JavaScript/Flash upload tool to upload files to the web app. To make it easier for users to upload large and/or remote files to the app, we also wanted to let them upload via SFTP. The catch was, our users didn't have SFTP accounts on our server and we didn't want to get into the business of creating and managing SFTP accounts. Enter: ProFTPD and virtual users.

ProFTPD's virtual users concept allows you to point ProFTPD at a SQL database for your user and group authentication. This means SFTP logins don't need actual system logins (although you can mix and match if you want). Naturally, this is perfect for dynamically creating and

  [Read more...]
Mailbox conversion
+0 Vote Up -2Vote Down

Converting from uw-mailboxes (mbx) to Unix format (dovecot)

It took me by surprise how the mailbox formats had changed, when I switched to Dovecot which is the best IMAP/POP3 mail program, in my opinion. It handles large (enormous) mailboxes with many (hundreds of) users. Caching makes things go fast again.

Here is a simple bash utility, to take all users, and convert all mailboxes & folders.

cd /home
for u in *
do
if [ -d /home/$u/Mail ]; then
echo "User: $u"
cd /home/$u/Mail

read more

Tools: What file system?
+1 Vote Up -2Vote Down

On Linux, you have a plethora of File Systems available.

Recently I was testing databases (MySQL, MariaDB) especially with ThreadPool against a Violin Flash Memory Array, and I forgot what file system the LUN was mounted (and formatted) as.

There are two ways to find out:

  • df -T (show mounted life systems, along with the FS type)
  • file -s /dev/xxx (show more details about a FS node)

read more

Net tools
+0 Vote Up -3Vote Down

A set of internet tools, is available at otala.net/tools.shtml

  • Traceroute
  • Whois & JWhois
  • NSLookup
  • Ping
  • DNSWalk
  • NetStat
  • ENUM (e164.org phone number lookup)

More to come, as (if/when) I get time -- suggestions welcome

What a Hosting Provider did Today
+0 Vote Up -0Vote Down
I found Dennis the Menace, he now has a job as system administrator for a hosting company. Scenario: client has a problem with a server becoming unavailable (cause unknown) and has it restarted. MySQL had some page corruption in the InnoDB tablespace. The hosting provider, being really helpful, goes in as root and first deletes ib_logfile* then ib* in /var/lib/mysql. He later says “I am sorry if I deleted it. I thought I deleted the log only. Sorry again.”  Now this may appear nice, but people who know what they’re doing with MySQL will realise that deleting the iblogfiles actually destroys data also. MySQL of course screams loudly that while it has FRM files it can’t find the tables. No kidding! Then, while he’s been told to not touch anything any more, and I’m trying to see if I can recover the deleted files on ext3 filesystem (yes there are tools  [Read more...]
What's New in CFEngine 3: Making System Administration Even More Powerful
+1 Vote Up -0Vote Down

CFEngine is both the oldest and the newest of the popular tools for automating site administration. Mark Burgess invented it as a free software project in 1993, and years later, as deployments in the field outgrew its original design he gave it a complete rethink and developed the powerful concept of promise theory to make it modular and maintainable. In this guise as version 3, CFEngine stands along with two other pieces of free software, Puppet and Chef, as key parts of enterprise computing. Along the way, Burgess also started a commercial venture, CFEngine AS, that maintains both the open source and proprietary versions of CFEngine.

Diego Zamboni has recently taken the position of Senior Security Advisor at CFEngine AS and is writing


  [Read more...]
Things I didn't know you could do with top...
+11 Vote Up -0Vote Down
My colleague Mike Hamrick, who I'm very fortunate to work with, made a presentation on some cool things you can do with top: http://tinyurl.com/48pmu83. These are some great tips that every MySQL DBA or developer should watch!
Keeping Up
+2 Vote Up -0Vote Down

I found I never published this post as it was sitting in my drafts few months now — it was written in 13th February, 2010. I’m publishing it without any changes.

I learn therefore I am!

I’ve just wrote few bits about learning a new technology and after skimming through my Google Reader, I noticed a great post by Chen Shapira — Deliberate Practice. That’s reminded me about another aspect of learning that I didn’t mention — learning is a continuous process.

There are two aspects…

  • No matter how good I am and how much I know, my knowledge and expertize become outdated relatively quickly these days unless I keep up with the
  [Read more...]
An SSH tool to make your life easier
+6 Vote Up -2Vote Down

A MySQL user group member saw that I use Poderosa as my ssh-on-Windows tool, and asked why I did not use PuTTY. My response was that I like having tabbed windows and hate having to keep opening another PuTTY program every time I want to open another connection. With Poderosa I can open a new connection with Alt-N, and I can even connect directly to Cygwin with an icon.

But Poderosa is not the tool I wanted to mention….Another user group member mentioned PuTTY Connection Manager. It wraps around PuTTY and gets the existing saved connections, makes a nicely tabbed browsing window where you can open sessions by double-clicking the connections, which are now listed on the right-hand side.

See screenshot below:

  [Read more...]
Liveblogging: Senior Skills: Sysadmin Patterns
+1 Vote Up -4Vote Down

The Beacon Pattern:
- This is a “Get out of the business” pattern
- Identify an oft-occurring and annoying task
- Automate and document it to the point of being able to hand it off to someone far less technical

Example:
- System admins were being put in charge of scheduling rooms in the building
- They wrote a PHP web application to help them automate the task
- They refined the app, documented how to use it, and handed it off to a secretary
- They have to maintain the app, but it’s far less work.

The Community Pattern:

- Prior to launch of a new service, create user documentation for it.
- Point a few early adopters at the documentation and see if they can use the service with minimal support
- Use feedback to improve documentation, and the service
- Upon launch, create a mailing list, forum,











  [Read more...]
Liveblogging: Seeking Senior and Beyond
+2 Vote Up -1Vote Down

I am attending the Professional IT Community Conference – it is put on by the League of Professional System Administrators (LOPSA), and is a 2-day community conference. There are technical and “soft” topics — the audience is system administrators. While technical topics such as Essential IPv6 for Linux Administrators are not essential for my job, many of the “soft” topics are directly applicable and relevant to DBAs too. (I am speaking on How to Stop Hating MySQL tomorrow.)

So I am in Seeking Senior and Beyond: The Tech Skills That Get You Promoted. The first part talks about the

  [Read more...]
Ravelry Runs On – 2010
+0 Vote Up -0Vote Down

I guess that it’s time for the 3rd annual “Ravelry Runs On” roundup. The last two were in March 2008 and March 2009.

This year, our traffic increased by 50% to 5,000,000 page views and 15 million Rails requests per day. We made very few changes to our architecture in 2009 but we did add a new master database server after our working set of data outgrew our memory and IO capacity.

This summary is more detailed then the last two and I’ve broken it up into rough sections.

Physical Network

We own our own servers and colocate then in a datacenter here in Boston. The datacenter provides us with a cooled

  [Read more...]
Perl CGI::param Overloaded Method?
+0 Vote Up -0Vote Down

This is a little story of a little bug. This gremlin suddenly appeared in a CGI.PM web-based application I work with. To make a long story short, an email was coming out something like this . . . 

389939
Subject:Update to Report #389939 by B. bloggins Description:389939 #389939: TPDD Now Deploying to monitoring for the MySQL servers.

 . . . when it should have been some thing like this:

Subject: TPDD Update to Report #389939 by B. bloggins

TPDD Now Deploying to monitoring for the MySQL servers.

After about an hour tracking things back, my team and I narrowed it down to this line of code:

$self->send_TXT_email(CGI::param("rep_no"),$rep_object,
                     $subject,$user_ref);

We scratched our respective



  [Read more...]
Free 10-day trial of Safari Books Online
+2 Vote Up -0Vote Down

That’s right — get your free 10-day trial! All the information I know is here:

http://bit.ly/37E9ld

But the basics are: No access to Rough Cuts or Downloads, for new subscribers only. It’s one of those “sign up and if you do not cancel after 10 days, we bill you” — and at $42.99 a month, that’s not a mistake you want to make. Must sign up by Nov. 24th.

To sign up now: https://ssl.safaribooksonline.com/tryitfree

I was asked to send this information along, so I am…Now’s your chance to skim High Performance MySQL, among other high quality books!

The Flipside of Uptime
+2 Vote Up -0Vote Down

We just had a booboo in one of our internal systems, causing it to not come up properly on reboot. The actual mishap occurred several weeks ago (simple case of human error) and was in itself a valid change so monitoring didn’t raise any concerns. So, as always, it’s interesting and useful to think about such events and see what we can learn.

Years ago, but for some now still, one objective is to see long uptime for a server, sometimes years. It means the sysadmin is doing everything right, and thus some serious pride is attached to this number. As described only last week in Modern Uptime on the Standalone Sysadmin blog, security patches are a serious issue these days, and so (except if you’re using ksplice

  [Read more...]
Possible Improvements to MySQL Administration
+2 Vote Up -0Vote Down

One thing you learn when you start to manage several servers is that life is easier if things are done the same way. I manage a few MySQL database instances on a few different hosts and here are a list of some things which I think could do with improvement. Note these views are from a UNIX point of view and for Windows MySQL DBAs many of the comments may not be appropriate. Also I’m not talking here about the administration of a single database, but the problems when you administer multiple instances.

Issues

First lets talk about the issues I notice in my day to day usage of MySQL. Then I’ll try and come up with some suggestions as to how these issues might be resolved.

  • Give up root earlier and expect to run as a non-root user. While it’s true that the mysqld process normally runs as the mysql user it’s also true
  [Read more...]
I’m Offering Pro-Bono Consulting
+1 Vote Up -0Vote Down

I started my company about a year ago, but I’ve been doing consulting for a long time. In fact, my first job in the IT industry was working for a consulting firm. Before that, starting as far back as grade school, I was involved in a lot of volunteer civic and community service activities. I admire companies who get involved in their communities, or even outside of their communities, wherever help is needed.

As part of my business plan, I’ve put in place a policy of accepting one pro-bono consulting project per year. So far, I haven’t gotten any requests for free consulting work, so here’s my public shout out to let you know what types of services are available:

1. Speaking or Training. My specialties are things like advanced Linux administration and SQL, but I’m perfectly capable of delivering

  [Read more...]
Using an empty database (learn from your mistakes)
+2 Vote Up -0Vote Down

I’ve been working on various different MySQL related issues and maintenance procedures some of which have not gone according to plan.  Here is a recipe that may help you avoid wasting a lot of time, especially if your database is large.

In order to do some of these tests make tests against a server configured identically to the one you plan to work on but instead which has no data. That is the mysql database needs to be complete but the other databases need to be dumped with the –no-data or -d options.  Don’t forget to also include any triggers or stored routines.

Now run the “procedure” on this “emtpy instance”. As it has no data most things run very quickly. So if you have issues you can repeat the procedure in no time. Restoring the instance too is easy as it’s tiny. This

  [Read more...]
Scalable Internet Architectures
+1 Vote Up -0Vote Down

My old friend and collaborator Theo Schlossnagle at OmniTI posted his slides from his Scalable Internet Architectures talk at VelocityConf 2009.

The slides are brilliant even without seeing Theo talk and I highly recommend the time it takes to flip through them, for anyone who is interested in systems performance. If anyone took an mp3 of this talk I’m dying to hear it, please let me know.

For those of you unfamiliar with OmniTI, Theo is the CEO of this rather remarkable company specializing in Internet-scale architecture consulting. They generalize on Internet-scale architecture, not on one specific dimension the way Pythian specializes on the database tier. This allows them to see Internet-scale workloads from a unique systemic, multidisciplinary point of view; from the user experience all

  [Read more...]
Book: Pro Linux System Administration
+0 Vote Up -0Vote Down

Peter Lieverdink (also known as cafuego on IRC/identi.ca, engineer on OurDelta builds and for Open Query) has co-authored a book that’s available since Monday. The title is Pro Linux System Administration published by Apress.

These days some people don’t want to bother with system administration, and either hire or outsource. Others want to find out more and do things themselves (home and small office use), and that’s the intended audience for this book.

How to Have a Good Presentation
+1 Vote Up -0Vote Down

In about 15 minutes, Giuseppe Maxia will begin a webinar in which the main focus is a presentation on “How to have a good presentation”. Talk about meta!

Giuseppe posted how to join the free webinar.

The slides can be found at http://datacharmer.org/downloads/2009_03_Presentation.pdf.

Teaching a Course on Profiling and Debugging in Linux
+0 Vote Up -0Vote Down

Dear Lazyweb,

So, I’ve been in Chicago for a week teaching a beginner and an intermediate course on using and administering Linux machines. This week, I’ll teach an intermediate and an advanced course on Linux, and the advanced course will cover profiling and debugging. The main tools I’m covering will be valgrind and oprofile, though I’ll be going over lots of other stuff, like iostat, vmstat, strace, what’s under /proc, and some more basic stuff like sending signals and the like.

So what makes me a bit nervous is, being that the advanced students are mostly CS-degree-holding system developers, they’ll probably be expecting me to know very low-level details of how things are implemented at  the system/kernel level. I’d love to know more about that

  [Read more...]
2009: Waiting to Exhale
+0 Vote Up -0Vote Down

Lots of blogs list a bunch of stuff that happened in the year just past, and I have done a year-in-review post before, but in looking back at posts on this blog and elsewhere, what strikes me most is not the big achievements that took place in technology in 2008, but rather the questions that remain unanswered. So much got started in 2008 — I’m really excited to see what happens with it all in 2009!

Cloud Computing

Technically, the various utility or ‘cloud’ computing initiatives started prior to 2008, but in my observation, they gained more traction in 2008 than at any other time. At the beginning of 2008, I was using Amazon’s S3, and testing to expand into more wide use of EC2 during my time as Technology Director for AddThis.com (pre-buyout). I was also investigating tons of other technologies that take different approaches to

  [Read more...]
Open Source Technology US Conference Calendar
+0 Vote Up -0Vote Down

One of the best ways to keep up with your field and network at the same time is to attend conferences. It’s one of the things I look forward to every year. After learning that O’Reilly has decided to commit blasphemy and *not* hold OSCON in Portland, Oregon the same week as the Oregon Brewers Festival, I was inspired to look around at what other conferences I might attend in 2009. Turns out, this is a huge pain in the ass, because I can’t find a single, central place that lists all of the conferences I’m likely to be interested in.

So… I created a public Google Calendar. It’s called “US Technical Conferences”. It needs more conferences, but I’ve listed the interesting ones I found. In order to keep the calendar from getting overwhelmingly crowded, I’ve decided that conferences on the list

  [Read more...]
How Are You Staffing Your Startup?
+0 Vote Up -0Vote Down

I have, in the past, worked for startups of varying forms. I worked for a spinoff that ultimately failed but had the most awesome product I’ve ever seen (neural networks were involved, need I say more?), I helped a buddy very early on with his startup, which did great until angel investors crept in, destroyed his vision, and failed completely to understand the Long Tail vision my buddy was trying to achieve, and I worked for a web 2.0 startup which was pretty successful, and was subsequently purchased… by another startup!

Working in academia for 6 years also exposed me to people who are firing up businesses, or projects that accidentally become businesses, and some of those go nowhere, while others seem to be on the verge of NYSE listing now, while a year ago they were housed in the smallest office I’ve ever seen, using lawn furniture for their

  [Read more...]
Help me pick a new feed reader
+0 Vote Up -0Vote Down

I’ve been using Google Reader since it was created. I really love the *idea* of Google Reader. I like that scrolling through the posts marks them as read. I like that you can toggle between list and expanded views of the posts. I like that you can search within a feed or across all feeds (though selecting multiple specific feeds would be great).

All of that said, I’d like to explore other avenues, because I don’t like that there’s, like, zero flexibility in how the Google Reader interface is configured. My problem starts with large fonts…

I use relatively large fonts. If you increase the font twice up from the default size in firefox on a mac (using the cmd-+ keystroke, twice), and you have more than just a couple of feeds, you wind up with this really horrible side pane with the bottom half of it requiring a scroll bar, and

  [Read more...]
MySQL Problem and Solution Posts: r0ck.
+0 Vote Up -0Vote Down

Taming MySQL is… challenging. Especially in very large, fast-growth, ‘always-on’ environments. It’s one of those things where you seemingly can never know all there is to know about it. That’s why I really like coming across posts like this one from FreshBooks that describes a very real problem that was affecting their users, how they dealt with it, why *that* failed, and what the final fix was. Post a link to your favorite MySQL Problem and Solution post in the comments (oh yeah, and “subscribe to comments” should be working now!)

I’m a Top 25 Geek Blogger… for some value of “Top”
+0 Vote Up -0Vote Down

I’m not someone who wakes up every day and looks at how my blog is ranked by all of the various services. I check out my WordPress stats, but that’s really about it. However, someone went and did some of the work for me, and they’ve decided that, of the blogs that they read or that were suggested to them, this blog ranks #20 in a listing of 25.

I’m really flattered, but wonder if it’s an indicator that this is a quality blog, or that they should aim higher in their blog reading ;-P  Either way, listing 25 bloggers in a flattering way is a fantastic marketing technique, because most of us are probably egomaniacal enough to say “Hey! Look!” and link back to the list on *your* blog, resulting in lots of traffic. Kudos, and thanks Mobile Maven!

Stop Doing Things That Don’t Work (a.k.a: Excel and Virtual Private Servers are Evil)
+0 Vote Up -0Vote Down

Note that I’m talking about using these tools in some kind of professional way, and more specifically, I’m talking about using Excel as a database, and using VPS hosting to host “professional” web sites. By “professional”, I mean something other than your personal blog, picture gallery, or other relatively inconsequential site.

Excel is not a database

Here’s the thing: Excel isn’t a database. Most people who don’t work in IT don’t seem to understand this, and they’re deathly afraid to actually communicate with anyone in IT, so they take matters into their own hands, and create problems so big that IT is forced to get involved, because at some point this spreadsheet becomes “critical” to some business function. Then IT gets even more

  [Read more...]
Generating Reports with Charts Using Python: ReportLab
+0 Vote Up -0Vote Down

I’ve been doing a little reporting project, and I’ve been searching around for quite some time for a good graphing and charting solution for general-purpose use. I had come across ReportLab before, but it just looked so huge and convoluted to me, given the simplicity of what I wanted at the time, that I moved on. This time was different.

This time I needed a lot of the capabilities of ReportLab. I needed to generate PDFs (this is not a web-based project), I needed to generate charts, and I wanted the reports I was generating to contain various types of text objects in addition to the charts and such.

I took the cliff-dive into the depths of the ReportLab documentation. I discovered three things:

  • There is quite a lot of documentation
  • ReportLab is quite a capable library
  • The documentation actually defies
  •   [Read more...]
    Showing entries 1 to 30 of 66 Next 30 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.