Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Previous 30 Newer Entries Showing entries 31 to 60 of 66 Next 6 Older Entries

Displaying posts with tag: Sysadmin (reset)

Clone a table in MySQL without killing your server
+0 Vote Up -0Vote Down

So, I recently ran into one of those situations where a customer complains that his MySQL database is slow, and “it worked great until about two weeks ago”. The first thing I look at in these situations is not the queries or any code, but the indexes. Once I determined that the indexes were almost nonsensical, I had a look at the queries themselves, which proved that the indexes were, in fact, nonsensical. Running the queries as written in the code, from a mysql shell, with EXPLAIN, I was able to further confirm that the indexes (most of them, anyway) were never used.

Easy right? Just reindex the table!

NOTE: I’m going to skip the talk about all of the database design issues on this gig and just go straight to solving the immediate problem at hand. Just know that I had nothing to do with the design.

  [Read more...]
Open Source Days 2008 , Day 2
+0 Vote Up -0Vote Down

As I was already up since yesterday 0500 , it was dinner with Sven , Robin and some other conference visitors at a Turkish Buffet place , after which we headed to what seemed to be a great bar where they failed to serve us while waiting for over 10 minutes, so we moved on to another place. and then to be "early"

After walking around a bit in Copenhagen and looking for a bus stop to go to the university I managed to bump into Wim & Co who offered me a ride to the IT University. Where I was almost in time for the first talk by
Jan Wieck about Slony-I, A master to multiple slaves-replication system for PostgreSQL
Given my recent MySQL MultiMaster setups I was fairly interested where PostgreSQL is at today.

Jan started out with explaining where he used replication the most,
For backups and Specialized services so he could offload long running and intrusive reporting tools



  [Read more...]
A merger, migration, mysql, python, and more news
+0 Vote Up -0Vote Down

First, AddThis.com (where I was the director of IT) and Clearspring have merged! A side effect of that is that I’m now (happily, on purpose, by choice) a full-time consultant! I’ll have a web site up soonish. Until then, check back here for updates. If you’re a tech firm who needs help, and don’t mind remote workers, send mail to bkjones at Google’s mail service (.com).

Some folks thought I’d passed away due to the uncharacteristic lull in posting frequency on this blog. I’m very much alive — but working for a startup and maintaining a consulting business simultaneously is hard, especially when two large projects fall into your lap at the same time. So what have I been up to?

Well, as

  [Read more...]
Sustained IO on EBS == No Bueno
+0 Vote Up -0Vote Down

I have a small EC2 instance running with a 25GB EBS volume attached. It has a database on it that I need to manipulate by doing things like dropping indexes and creating new ones. This is on rather large (multi-GB, millions of rows) tables. After running one DROP INDEX operation that ran all day without finishing, I killed it and tried to see what was going on. Here’s the results of the first 10 minutes of testing:

-bash-3.2# dd if=/dev/zero of=/vol/128.txt bs=128k count=1000
1000+0 records in
1000+0 records out
131072000 bytes (131 MB) copied, 0.818328 seconds, 160 MB/s

This looks great. I’d love to get 160MB/s all the time. But wait! There’s more!

-bash-3.2# dd if=/dev/zero of=/vol/128.txt bs=128k count=100000
dd: writing `/vol/128.txt': No space left on device
86729+0 records in
86728+0 records out
11367641088 bytes (11 GB) copied,
  [Read more...]
More Adventures in Amazon EC2 (and EBS)
+0 Vote Up -0Vote Down

Short Version: You can find a fantastic video here about bundling customized AMIs and registering them with Amazon so that you can launch as many instances of your new AMI as you want. The video is so good that I don’t bother writing out the steps to do the bundling (it would be pretty darn long). These are some notes about launching an AMI, customizing it, and mounting an EBS volume to it (the video linked above doesn’t cover EBS). Also, check out the ElasticFox tool which is a very good GUI for doing simple EC2 operations. Nice if you’re just getting started or doing some simple tests.

There are two ways you can go about creating a custom machine image (AMI) for use with Amazon EC2: You can create an image locally by

  [Read more...]
PyWorks Conference Schedule Posted
+0 Vote Up -0Vote Down

Hi all,

The schedule for PyWorks has been posted! I’m really excited about three things:

1) there are some really cool talks that I’m looking forward to attending. There are a couple of sysadmin-related talks, AppEngine, TurboGears, Django, and an area I’ve been especially slow to move into: testing (I know, shame on me). There’s lots more so be sure to check it out.

2) the conference scheduling process is over

3) I get to meet a lot of people face-to-face that I’ve worked with in the past on Python Magazine developing articles, or interacted with on IRC, etc. One thing I like about conferences surrounding open source technologies is you get to thank people face-to-face for the

  [Read more...]
Why I Don’t Write Book Reviews
+0 Vote Up -0Vote Down

I have a lot of interaction with publishing types. I write a lot, and I edit some, and I do tech reviews and stuff for some publishers, and I co-authored a book, and I’ve worked on two magazines, and a newspaper, and I’m generally fascinated by the technical book market and stuff like that. I’m also someone who is lucky enough that his job is also his hobby. I work in technology, and am always doing something technology related at home in my spare time. Needless to say, I read tons upon tons of technical books.

I almost never post book reviews, in spite of the fact that I read all of these books. Why? Well, to be honest, I couldn’t tell you. It just hasn’t occurred to me to write a book review. Could be because I don’t really value book reviews too much myself I guess. I mean, if there’s a really obvious consensus across a huge

  [Read more...]
Next week, meet me in Frankfurt, Stuttgart, Munich or Prague!
+0 Vote Up -0Vote Down

I am traveling to Europe next week to brief major prospects in Germany (Daimler, MAN) as well as to attend to administrative matters at Pythian Europe in Prague and would love to meet any readers of this blog during this trip!

I’m especially interested in meeting:

  • DBAs, Applications Administrators and Systems Administrators,
  • Potential customers (IT Directors, DBA Managers, Supply Managers for IT), and
  • Potential partners (IT product of service companies that could partner with Pythian to delight our mutual customers)

Here is my itinerary:

  • Sunday, August: Frankfurt,
  • Monday, August 4: Stuttgart,
  • Tuesday, August 5: Munich, and
  • Wednesday, August 6 through Saturday, August 9: Prague, Czech Republic.

Please reach out to me using vallee@pythian.com if you would like to meet!

The promise of Drizzle
+0 Vote Up -0Vote Down

I got to actually speak to Brian Aker for maybe a total of 5 minutes after his micro-presentation about Drizzle, which took place at the Sun booth at OSCON 2008. I was a bit nervous to ask what questions I had out loud, because the things I had wondered about were things I really didn’t see too much discussion about out in the intarweb. I’m happy to report that, if Brian Aker is to be considered any kind of authority (hint: he is), my ideas are not completely ridiculous, so maybe I’ll start talking a bit more about them.

UPDATE: lest anyone get the wrong idea, Brian Aker did, in fact, state that views are not on the short list of priority items for Drizzle, but he did say that views are one of the features he finds most useful, and that they’d probably

  [Read more...]
OSCON Day 2: Launching a Startup in 3 Hours
+0 Vote Up -0Vote Down

Launching a Startup in 3 Hours was a great talk given by Andrew Hyde (of techstars.org) and Gavin Doughtie (of Google). Both of the speakers are heavily involved in the recent trend of doing “Startup Weekends”, and techstars.org is an organization that hosts startup weekends all around the US (and I think internationally as well - Andrew mentioned one in Germany if I heard correctly).

The first half of the talk was about the general concept of a startup weekend, the problems it avoids (”we’ve been working for 9 months and haven’t launched anything”), the problems it brings up (”If you’re not using Java, you’re an idiot, so count me out!!”), and lots of details about how to organize, how to assign roles, and some common tools they use (like Basecamp and whatever your IM of choice is). There was also

  [Read more...]
OSCON Evening 1 Begins, and More Portland Tips
+0 Vote Up -0Vote Down

The evening plans didn’t wait for talks to be done. The IRC channel (#oscon on irc.freenode.net) was alive with talk of prospects for dinner and drinks after the conference. I myself was torn between a group going out for Lebanese and another going to Henry’s, but opted to go with my buddies from home to Henry’s.

It was worth it. If you haven’t been, Henry’s Tavern boasts 100 beers and hard ciders on tap (oddly, the beer list is the only menu *not* online - guess it changes too frequently). There are a ton of local beers that you can’t even get on the east coast just waiting for you to try, but there are also some rare treats, like the Belgian Lambic beers, which you don’t often see on tap. The food is a little pricey, but is really good, and the staff is very

  [Read more...]
OSCON Day 1 Comes to a Close
+0 Vote Up -0Vote Down

I think I have pictures of most of the basic parts of the conference at my OSCON Flickr set, and I thoroughly enjoyed day 1 of the conference. Of course, while *day* 1 is over, *night* 1 has yet to even begin. There are lots of BoF sessions, and maybe even more smaller meetups going on, as smaller groups take to discussing things over dinner and a beer or three.

I have to say, that I occasionally pop into irc channels for conferences I’m not even at and follow up on that because I’m involved a bit in conference planning as part of my work with Python Magazine (I’m helping to organize the PyWorks conference in November). This conference seems to have a pretty happy audience, if IRC chatter is any indication (and it usually is). Sure, there are a couple of weak spots in the wireless

  [Read more...]
Day 1 of OSCON Begins, and More Tips for Conference-goers
+0 Vote Up -0Vote Down

I got an early start. Too early. But I’m from the west coast, so my body thinks I slept in. I wandered around a bit, took a few pics which you can see at my Flickr OSCON set, and I discovered a couple of things that might be of interest:

  • The starbucks in the conference center charges over $2 for a small cup of joe. There’s a starbucks right across the street (you can see it from the breakfast area - seriously, it’s 5 seconds away), and they charge less than $2 for a medium (grande). That’s less than I pay at home.
  • The ATM outside the starbucks charges $3 for cash. I’ll report back when I find a cheaper one, but most places seem to take plastic here.
  • Every computer involved in this conference, from registration to the
  [Read more...]
Useful stuff - 2008 - first half
+0 Vote Up -0Vote Down

Having a Google account is sometimes useful in ways you hadn’t planned for. For example, at a few different employers I’ve been at, I’ve had to prepare for reviews by providing a list of accomplishments to my supervisor. One decent tool for generating this list is email, though it can take some time. Another useful tool is the Web History feature of your Google account.

Though this isn’t necessarily indicative of everything I’ve accomplished in the first half of 2008 per se, it’s definitely indicative of the types of things I’ve generally been into so far this year, and it’s interesting to look back. What does your Web History say?

  • Gearman - this is used by some rather large web sites, notably Digg. It reminds me a little of having Torque and Maui, but geared toward
  [Read more...]
Cloud computing hype overload
+0 Vote Up -0Vote Down

I’ve been working with what I used to call “utility computing” tools for about 6-9 months. However, for about the past 2 months, I’ve been seeing the term “cloud computing” all over the place, and there is so much buzz surrounding it that it’s reaching that magical point best described using Alan Greenspan’s words: “Irrational Exuberance”.

When Alan Greenspan used those words to describe the attitudes of investors toward the markets, what he was basically saying was that there were people who didn’t really know what they were doing, putting more money than they ought, into things they knew relatively little about. Further, he was saying that the decisions people were making with regards to where to put their money were a) bad, or at least b) not based on sound reasoning, or the ‘facts on the

  [Read more...]
Plug-ins: isn?t there a better way?
+0 Vote Up -0Vote Down

If there’s one thing that bothers me about using a ready-made solution like wordpress for my blog, it’s plug-ins. I hate software plug-ins. The first question every support engineer for any software product that supports plugins asks in response to a trouble report is “are you using any plugins?” And when you say “yep, I’m using plugins!” the reply from support is to disable them immediately and see if the trouble goes away. That’s a problem.

What’s worse, if the plugins are maintained by a third party (often the case), there’s no telling whether or not they’ll exist when the next version of the base software is released, or whether they’ll be

  [Read more...]
Spacewalk, and what we can learn about naming
+0 Vote Up -0Vote Down

Red Hat releases Spacewalk. It is described as: “the upstream community project from which the Red Hat Network Satellite product is derived“. Congratulations to all whom have worked on it, especially my friends who tired endlessly over it in the past.

Red Hat, is sticking true to its promise, of open sourcing everything they make. Best of all, they recognise Fedora (they always did, since say, Fedora Core 2 or 3), CentOS (a direct “competitor”/rebuild of RHEL), and Scientific Linux (I know of a certain university’s sysadmin who will be blessing Spacewalk, as her life will now be a lot easier).

There have been a few blogs about it… Matt Asay asks about a community (Red Hat traditionally

  [Read more...]
Why should I pay for this AWS design decision?
+0 Vote Up -0Vote Down

I was writing a utility in Python (using boto) to test/play with Amazon’s SQS service. As boto isn’t particularly well documented where SQS specifically is concerned, I also plan to post some examples (either here or on Linuxlaboratory.org, or both). When I had some trouble getting a message that was sent to a queue, I went to the Amazon documentation, and found this little gem in the Amazon Web Services FAQ

I am sure that my queue has messages, but a call to ReceiveMessage returned none. What could be the problem?

Due to the distributed nature of the queue, a weighted random set of machines is sampled on a ReceiveMessage call. That means only the messages on the sampled machines are

  [Read more...]
O?Reilly: Give your authors Safari access!
+0 Vote Up -0Vote Down

UPDATE - 2008-06-23 - A member of O’Reilly’s editing team commented that this privilege has *NOT* been discontinued, and all O’Reilly authors should receive a free Safari account. Thanks a bunch, Mary, for the clarification (see comments for more).

I learned from one of the authors of the recently released second (read: first, squared) edition of High Performance MySQL that O’Reilly apparently did away with the idea of giving O’Reilly book authors free Safari accounts. Lame.

I do not know why in the world they would discontinue this offering for authors. Perhaps they’re not aware, but a great many of the O’Reilly authors are also bloggers. Tech

  [Read more...]
High Performance MySQL on Safari!
+0 Vote Up -0Vote Down

All right! In the past, some books seem to be delayed in getting into O’Reilly’s Safari site, but on the day that Baron announces the book’s arrival, I find that I’m able to access it in Safari right now! Sweet!

addthis_url = 'http%3A%2F%2Fwww.protocolostomy.com%2F2008%2F06%2F20%2Fhigh-performance-mysql-on-safari%2F'; addthis_title = 'High+Performance+MySQL+on+Safari%21'; addthis_pub = 'jonesy';
For anyone flying to a conference - flyer beware
+0 Vote Up -0Vote Down

I’m going to OSCON in July, and I know that just about everyone I know who is a participant in this crazy life we call IT (or web 2.0, or whatever it’s called now), is flying to a conference or something in 2008. I’m starting to notice more and more posts like this one, so if you can avoid it, don’t put anything in a checked bag that you can’t afford to lose, and avoid US Airways, and pass it on, because when you see the list of things they don’t cover in their lost baggage policy, you’ll suddenly feel like you’re lucky to still have anything you ever checked with your bags.

addthis_url = 'http%3A%2F%2Fwww.protocolostomy.com%2F2008%2F06%2F13%2Ffor-anyone-flying-to-a-conference-flyer-beware%2F'; addthis_title = 'For+anyone+flying+to+a+conference+-+flyer+beware'; addthis_pub = 'jonesy';
MySQL Data Sharding Toolkit in Python
+0 Vote Up -0Vote Down

This rocks. It’s not complete, but Pyshards is the closest thing I’ve seen to a real attempt at making a more or less generic sharding toolkit, written in Python. This is not just great because it’s written in Python or because it helps people who need sharding capabilities in MySQL. It’s great because having a toolkit to use for this benefits the community by creating a point of reference for how to get things done, and can help unite those who are treading into this territory and help them all get a leg up on this beast that is “sharding”.

I, for one, have found ways (so far) to avoid having to do this. It’s a good bit of complexity for data that would otherwise be very simple, and an infrastructure architecture that would otherwise also be simple (by

  [Read more...]
Notes on Book Shopping from a Tech Bibliophile
+0 Vote Up -0Vote Down

Hi. My name is Brian, and I’m a tech bibliophile.

I have owned more books covering more technologies than I care to admit. Some of my more technical friends have stood in awe of the number of tech books I own. I am also constantly rotating old books that almost *can’t* be useful anymore out of my collection because there’s just no room to keep them all, and it would be an almost embarrassingly large collection if not for the fact that I have no shame or guilt associated with my need for dead trees.

If you need further proof:

  • I have, on more than one occasion, suggested to my wife that we take a walk around our local mall so I could browse the computer section of the book store, not to buy, but just to keep up with the new titles and stuff.
  • Ok, I usually buy.
  • I also go into book stores whenever
  [Read more...]
Scalability Best Practices: eBay
+0 Vote Up -0Vote Down

Following a link from the High Scalability blog, I found this really great article about scalability practices, as told by Randy Shoup at eBay. Randy is very good at explaining some of the more technical aspects in more or less plain English, and it even helped me find some wording I was looking for to help me explain the notion (and benefits) of functional partitioning. He also covers ideas that apply directly to your application code, your database architecture (including a little insight into their sharding strategy), and more. Even more about eBay’s architecture can be found here.

addthis_url = 'http%3A%2F%2Fwww.protocolostomy.com%2F2008%2F05%2F29%2Fscalability-best-practices-ebay%2F'; addthis_title = 'Scalability+Best+Practices%3A+eBay'; addthis_pub = 'jonesy';
Multisourced Production Infrastructure: History, and a stab at the Future
+0 Vote Up -0Vote Down

Startups are pretty fascinating. I work for a startup, and one of my good friends works for another startup. I’ve also worked for 2 other startups, one during the first “bubble”, and another one a few years later. Oh my, how the world of web startups has changed in that time!

1999: You must have funding

The first startup I was ever involved in was a web startup. It was an online retailer. They were starting from nothing. My friend (a former coworker from an earlier job) had saved for years to get this idea off the ground. He was able to get a few servers, some PCs for the developers he hired, and he got the cheapest office space in all of NYC (but it still managed to be a really cool space, in a way that only NYC can pull off), and he hosted every single service required to run the web site in-house. If

  [Read more...]
A Couple of MySQL Performance Tips
+0 Vote Up -0Vote Down

If you’re an advanced MySQL person, you might already know these, in which case, please read anyway, because I still have some questions. On the other hand, f you’re someone who launched an application without a lot of database background, thinking “MySQL Just Works”, you’ll eventually figure out that it doesn’t, and in that case, maybe these tips will be of some use. Note that I’m speaking specifically about InnoDB and MyISAM, since this is where most of my experience is. Feel free to add more to this content in the comment area. 

InnoDB vs. MyISAM

Which one to use really depends on the application, how you’re deploying MySQL, your plans for growth, and several other things. The very high-level general rule you’ll see touted on the internet is “lots of reads, use MyISAM; lots of writes,


  [Read more...]
MySQL Conf08 - My Interview with Jennifer Venable of Red Hat
+0 Vote Up -0Vote Down

Last Tuesday when I was walking the show floor at the MySQL conference, I ran into a familiar name,  Jennifer Venable.  I had never met Jennifer before but we had traded mails and had spoken on the phone.  This was about 18 months ago when we were negotiating the renewal of Sun's contract as an Authorized Distributor of Red Hat.  At that time, Jennifer was the Red Hat lawyer working on the contract.

Well since that time Jennifer has escaped from the Red Hat legal ranks and has joined the business side where, as of a couple of months ago, she took over as head of the Red Hat Exchange. 

Take a listen.

My interview with Jennifer (12:25)  Listen (


  [Read more...]
Is this thing on?
+0 Vote Up -0Vote Down

Since the recovery from my recent outage, I’ve noticed that none of the normal feed sites where my posts normally show up caught the last post, so this is a test post to see what’s going on, if it was a temporary glitch, or what.

If you didn’t see the post linked above, please read it if you’re happy with your web host. I’m looking for a new one :-/

addthis_url = 'http%3A%2F%2Fwww.protocolostomy.com%2F2008%2F04%2F16%2Fis-this-thing-on%2F'; addthis_title = 'Is+this+thing+on%3F'; addthis_pub = 'jonesy';
Need a new web host
+0 Vote Up -0Vote Down

Ok, this blog is currently hosted by 1and1.com, and I think that needs to change, which is sad, because up until recently, I’ve been pretty happy with the performance. However, I recently had an issue, and some things came to light about my package that I wasn’t aware of.

First of all, the maximum time a process can run is about 6 seconds. Second of all, the maximum number of processes you can have running at once is 12. When I asked if this was a limitation of my *package*, the answer I got was that it was a limitation of all shared hosting accounts.

This blog started throwing 500 errors some time yesterday. I called support late last night and some lady said that she was running a tool that should fix my issue, and to call back in 30 minutes if the problem persists. Well, I’m sick as a dog, and it was late, so I went to bed. This morning, the problem was

  [Read more...]
Saying What You Mean
+0 Vote Up -0Vote Down

Ah, the perils of working in a shared, client environment. One client has us using a login that is not exclusive to us. I prefer using bash; the client is set to use zsh. This is not a problem in and of itself.

However, there is a section in the .profile that is causing me issues:

if [ -f /usr/bin/ksh ]; then
        /usr/bin/ksh -o vi
        exit
fi

So, “If ksh exists, run it with some options to edit history with vi-like commands”. Except what we really want is “If you’re using the ksh as a shell, . . . .”

So I added a modification, and now all is fine.

if [ -f /usr/bin/ksh ]; then
        if [ "$SHELL" = "/usr/bin/ksh" ]; then
                /usr/bin/ksh -o vi
                exit
        fi
fi

(not all my problems are MySQL related!)

Previous 30 Newer Entries Showing entries 31 to 60 of 66 Next 6 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.