Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 54 Next 24 Older Entries

Displaying posts with tag: Tech (reset)

From MySQL to MailChimp via CSV
+0 Vote Up -0Vote Down

Don't you hate disclaimers? I do, but before I do anything else, I must ask that you don't use the techniques below unless you are emailing responsibly.

Today I needed to pull email addresses for people who had signed up to a thing out of MySQL and into MailChimp so that I could actually email them about the thing. MySQL actually has a very cute feature for exporting the results of an SQL query as a CSV file, which I had to look up to remember how to do it. It goes something like this:

My original query*

SELECT u.username, u.email, a.account_name
FROM users u
INNER JOIN accounts a ON a.user_id = u.id
WHERE a.status_id = 2;

To write this to a file, we use MySQL's SELECT ... INTO OUTFILE feature, so my query

  [Read more...]
Handling Database Failover at Craigslist
+2 Vote Up -0Vote Down

There has been some interesting discussion on-line recently about how to handle database (meaning MySQL, but really it applies to other systems too) failover. The discussion that I’ve followed so far, in order, is:

  [Read more...]
SQL JOINing a Table to Itself
+1 Vote Up -0Vote Down

Getting two sets of information from one table in a select statement often leads people to write subselects, but it really doesn't matter that this is the same table twice, we can just give it a new alias and treat it as if it were a different table. This is one of those techniques where, once you've seen it, it's really obvious, but until that point it can be very confusing. I explained this to someone else recently, so I thought I'd capture it here in case it's helpful to anyone else.

Consider that tried-and-tested example: employees and managers. Here's the staff table from the database (today's imaginary data isn't particularly imaginative, sorry):

mysql> select * from staff;
+----+------------+-----------+------------+
| id | first_name | last_name | manager_id |
  [Read more...]
Linux Documentation Writer Wanted!
Employee +1 Vote Up -0Vote Down

The Oracle Linux and Virtualization Documentation Team is seeking an experienced Technical Writer
with a focus on writing documentation for the Oracle Linux product. (The MySQL Documentation Team is part of that group as well.)

Applicants should be located in either Ireland, the UK, Sweden, Norway,


  [Read more...]
Linux Documentation Writer Wanted!
Employee +0 Vote Up -0Vote Down

The Oracle Linux and Virtualization Documentation Team is seeking an experienced Technical Writer
with a focus on writing documentation for the Oracle Linux product. (The MySQL Documentation Team is part of that group as well.)

Applicants should be located in either Ireland, the UK, Sweden, Norway,


  [Read more...]
SQL Joins with On or Using
+1 Vote Up -0Vote Down

I recently wrote a post about inner and outer joins, and a couple of people asked what the difference is between USING and ON.

In a nutshell, you use ON for most things, but USING is a handy shorthand for the situation where the column names are the same.

Consider this example dataset:

mysql> select * from pets;
+---------+---------+--------+-----------+
| pets_id | animal  | name   | owners_id |
+---------+---------+--------+-----------+
|       1 | fox     | Rusty  |         2 |
|       2 | cat     | Fluffy |         2 |
|       3 | cat     | Smudge |         3 |
|       4 | cat     | Toffee |         3 |
|       5 | dog     |
  [Read more...]
Inner vs Outer Joins on a Many-To-Many Relationship
+1 Vote Up -0Vote Down

Someone will probably tell me that this is an elementary-level topic, but I got some good questions regarding joins from my most recent ZCE class students, so I thought I'd put down the examples that I used to explain this to them. Being able to join with confidence is a key skill, because it means that you can refactor and normalise your data, without worrying about how hard something will be to retrieve.

The database structure I'm using here looks like this (the same example that I used when I wrote the Databases chapter for PHP Master):



  [Read more...]
Explaining MySQL's EXPLAIN
+0 Vote Up -0Vote Down

The MySQL explain plan is a great tool to help developers and database administrators to improve the performance of specific queries happening against a database. It is very easy to use, but its output can be confusing, so I thought I'd show a very simple example.

Let's begin by looking at a table called recipes

mysql> desc recipes;
+-------------+--------------+------+-----+---------+----------------+
| Field       | Type         | Null | Key | Default | Extra          |
+-------------+--------------+------+-----+---------+----------------+
| id          | int(11)      | NO   | PRI | NULL    | auto_increment |
| name        | varchar(400) | NO   | MUL | NULL    |                |
| description | text         | YES  |     | NULL    |                |
| category_id | int(11)
  [Read more...]
Welcome to Insatiable Demand
+0 Vote Up -0Vote Down

In early 2006 Paul Hurley (ideeli’s CEO) and I (Mark Uhrmacher, CTO) were thinking about a new business. We had the idea to create a community based around great deals for Women’s fashion products where we saw a great deal of potential for great content and product sales. Now, over five years later, we’ve realized much of that vision. Our business success has been chronicled over the years in several places (see here and here). Though we’re very proud of our achievements there, that isn’t what this blog is about.

Insatiable Demand is about a mostly untold story. Over the past five-plus years we’ve built a phenomenal technology platform and team. From two people and three servers to a

  [Read more...]
Databases 10 Years Ago
+0 Vote Up -0Vote Down

In preparation for the talk I’m giving tomorrow, I was assembling a slide that looks back about 10 years ago and remembering what our database infrastructure was like back then. I was at Yahoo! during that time and our leading-edge MySQL deployments were on machines that just under 1GHz processors running a 32bit operating system, with relatively small, slow, and expensive disks (no SSDs!).

Back then MySQL 3.23 was the norm, though brave folks like us were running MySQL 4.0 beta releases in production to take advantage of the new and improved replication. Remember when simply having replication was a big deal?!

And InnoDB was really, really new–not for the faint of heart.

Oh, how times have changed.

Nowadays, I could probably simulate our old Yahoo! Finance

  [Read more...]
Using Persistent Storage with Gearman
+0 Vote Up -0Vote Down

I'm using gearman for the first time in a new project, and two things in particular were bothering me. Firstly, there doesn't seem to be a built-in way to see what's in the queue. Secondly, if the gearman server dies (which seemed quite likely when I was first getting to grips with this stuff and writing really buggy code!) you lose your queue. Therefore I decided that I would switch gearman over to running with persistent storage. My config file (this is Ubuntu 10.10) is in /etc/default/gearman-job-server and it contains the following snippet:

# Use mysql as persistent queue store                                                                                                  
# PARAMS="-q libdrizzle --libdrizzle-host=10.0.0.1 --libdrizzle-user=gearman \
  [Read more...]
Slides for: Fusion-io and MySQL 5.5 at Craigslist
+0 Vote Up -0Vote Down

The slides from the talk I gave at Percona Live San Francisco yesterday are now available on slideshare (hopefully the embed works here too):

View more presentations from Jeremy Zawodny.

Overall, I enjoyed the conference–not just meeting up folks I don’t see often enough, but also getting a renewed sense of how active the MySQL ecosystem really is.


When an example falls in your lap
+0 Vote Up -0Vote Down

As I recently noted, I’m giving a short talk at Percona Live about our experience with Fusion-io for MySQL at Craigslist. As is often the case, I agreed to give the talk before giving too much thought about exactly what I’d say. But recently I’ve started to sweat a little at the prospect of having to think up a compelling and understandable presentation.

Thankfully, due to a cache misconfiguration this week, we ended up taking a number of steps that not only will help us to deal with future growth, but as a side-effect we got to directly quantify some of the benefits of Fusion-io in our big MySQL boxes. For whatever reason, the bulk of the presentation basically fell into my lap today.

Now I just have to put it all together.

I won’t go so far as to claim that

  [Read more...]
Speaking at Percona Live in San Francisco
+0 Vote Up -0Vote Down

On Wednesday, February 16th, I’ll be attending Percona Live in San Francisco to hear about what’s new in the MySQL ecosystem and talk about our adoption of Fusion-io storage for some of our systems at Craigslist. Not only do we have a busy web site, the data itself has posed some unique challenges over the last few years.

Part of getting a handle on that was upgrading to faster storage and moving from years-old MySQL 5.0.xx to more modern releases. I’ll also provide a bit of background on our plans to continue scaling and growing in the coming years.

If you’re in the area and interested in some of the cutting edge work that’s been going into production as part of major MySQL/XtraDB deployments,

  [Read more...]
Is Enum Evil?
+0 Vote Up -0Vote Down

When I work on database designs, either on my own projects or as advisor to others, I often find people very reluctant to use an enum type for any columns. Now, I'm not about to advocate the gratuitous use of an enum column, they definitely have some pitfalls, but I think it is important to understand these rather than just shouting "enum evil!" if anyone should mention them.

There are cases where an enum is the correct choice for a particular type of data, so let's look at what an enum type actually is and does.


Enum Type Intentions

An enum column type is a column which allows only the specified data values. This means that if a particular column can only take certain values, for example 'art', 'music' or 'sport' then you can specify this


  [Read more...]
Always Test with Real Data
+0 Vote Up -0Vote Down

As I previously noted, I’m in the midst of converting some data (roughly 2 billion records) into documents that will live in a MongoDB cluster. And any time you move data into a new data store, you have to be mindful of any limitations or bottlenecks you might encounter (since all systems have had to make compromises of some sort or another).

In MySQL one of the biggest compromises we make is deciding what indexes really need to be created. It’s great to have data all indexed when you’re searching it, but not so great when you’re adding and deleting many rows.

In MongoDB, the thing that gets me is the document size limit. Currently an object stored in MongoDB cannot be larger than 4MB (though that’s likely to be raised soon). Now, you can build your

  [Read more...]
MySQL 5.5.4-m3 in Production
+0 Vote Up -0Vote Down

Back in April I wrote that MySQL 5.5.4 is Very Exciting and couldn’t wait to start running it in production. Now here we are several months later and are using 5.5.4-m3 on all the slaves in what is arguably our most visible (and one of the busiest) user-facing cluster. Along the way we deployed some new hardware (Fusion-IO) but not a complete replacement. Some boxes are Fusion-io, some local RAID, and some SAN.  We have too many eggs for any one basket.

We also converted table to the Barracuda format in InnoDB, dropped an index or two, converted some important columns to BIGINT UNSIGNED and enabled 2:1 compression for the table that has big chunks of text in it. Aside from a few false starts with the Barracuda conversion and compression, things went pretty well. Coming from 5.0 (skipping 5.1

  [Read more...]
Database Drama
+0 Vote Up -0Vote Down

There’s been a surprising amount of drama (in some circles, at least) about database technology recently.  I shouldn’t be surprised, given the volume of reactions to the I Want a New Datastore post that I wrote. (Hint: I still hear from folks pitching the newest data storage systems.)

The two things that caught my eye recently involve Cassandra and MongoDB (and, indirectly, MySQL). First was what I read as a poorly thought out and whiny critique of MongoDB’s durability model: MongoDB Performance & Durability. Just because something is the default doesn’t mean you have to use it that way. Thankfully there was reasoned discussion and reaction elsewhere, including the

  [Read more...]
Introduction to memcached
+0 Vote Up -0Vote Down

These are the slides to a talk I did earlier this week for students of the professional bachelor in ICT course at KaHo St. Lieven. I wanted to give a clear and simple introduction to the memcached service, as I think it’s an invaluable tool in today’s web development.

MongoDB Early Impressions
+0 Vote Up -0Vote Down

I’ve been doing some prototyping work to see how suitable MongoDB is for replacing a small (in number, not size) cluster of MySQL servers. The motivation for looking at MongoDB in this role is that we need a flexible and reliable document store that can handle sharding, a small but predictable write volume (1.5 – 2.0 million new documents daily), light indexing, and map/reduce operations for heavier batch queries. Queries to fetch individual documents aren’t that common–let’s say 100/sec in aggregate at peak times.

What I’ve done so far is to create a set of Perl libraries that abstract away the data I need to store and provide a “backend” interface to which I can plug in a number of modules for talking to different data stores (including some

  [Read more...]
My MySQL wishlist (revised)
+1 Vote Up -1Vote Down
I wrote about my MySQL wishlist on November 14th 2007 and now it's time for an update. I will copy-paste the old entry. The original text will be in italics.

1. Per user and/or per database quota
Would very useful in setups for shared hosting. This would also prevent one database from bringing down the whole server. Separate tablespaces on different mountpoint can ease the pain, but I consider that a nasty hack.

No update. Still problematic

2. External authentication
I've seen numerous scripts which fetch the authentication info from ldap, a file, another database or some other authentication store. This should be integrated into mysql. The mysql grant tables should be pluggable so it is possible to write a custom authentication plugin. We already have plugable engines and function (UDF) so this shouldn't be






  [Read more...]
Using the MySQL Test Suite
+0 Vote Up -1Vote Down
Earlier I reported about two crashes related to MySQL 5.0.22 on Ubuntu 6.06 LTS.

I think those bugs show a lack of testing on the side of Cannonical/Ubuntu. And for MySQL there is a quite good test suite available, so it's not rocketsience.

There are multiple reasons why you could use the MySQL Test Framework:
1. Test if bug you previously experienced exists in the version you are using or planning to use.
2. Test if configuration changes have a good or bad result on the stability of mysqld.
3. Test if important functions still return the correct results (especially importand for financial systems)

$ echo "SELECT @@version;" > version.test
$ cp version.test version.result
$ mysql < version.test >> version.result
$ mysqltest --result-file=version.result











  [Read more...]
Another Crash in MySQL 5.0.22 on Ubuntu 6.06 LTS
+0 Vote Up -7Vote Down
1. Set this variable
thread_stack = 265K

2. Execute this query
mysql> SELECT 0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0
+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+
0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0
+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+
0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0
+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+
0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0
+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+
0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0+0











  [Read more...]
Crash in MySQL 5.0.22 on Ubuntu 6.06 LTS
+0 Vote Up -2Vote Down
I found a new crasher in the MySQL 5.0 version which ships with Ubuntu 6.06 LTS.

> SELECT * FROM (SELECT mu.User FROM mysql.user mu UNION SELECT mu.user FROM mysql.user mu ORDER BY mu.user) a;
ERROR 2013 (HY000): Lost connection to MySQL server during query

The bug report: LP392236

On MySQL 5.0.51 on Debian stable it returns this error (like it should):
ERROR 1054 (42S22): Unknown column 'mu.user' in 'order clause'

The correct query should be like this (Using culumn a number):
> SELECT * FROM (SELECT mu.User FROM mysql.user mu UNION SELECT mu.user FROM mysql.user mu ORDER BY 1) a;
MySQL scanning module for Metasploit
+0 Vote Up -0Vote Down
I've created a very simple MySQL scanning module for the metasploit framework.

1. Download the mysql_version file and rename it to mysql_version.rb and put it in the framework-3.2/modules/auxiliary/scanner/mysql directory of your metasploit installation.
http://compukid.no-ip.org/dev/scripts/mysql_version

2. Use using msfcli
./msfcli auxiliary/scanner/mysql/mysql_version RHOSTS=192.168.0.1 E
[*] 192.168.0.1:3306, MySQL server version: 5.0.81-1-log (Protocol 10)

3. More options:
set THREADS to 10 and RHOSTS to 192.168.0.0/24 to scan a whole network.
Hibernate Wars: The Query Cache Strikes Back
Employee +1 Vote Up -0Vote Down

Not so long ago, in a galaxy not very far away... a middle-aged programmer battled to free memory from the dark side of the hibernate query cache.  He was successful, or so it seemed. For the query cache memory problems had risen from the ashes -- stronger and more bloated than ever...


What's With All This Garbage?


We hit a case at work (http://www.mysql.com/products/enterprise/monitor.html) (again), where the java server process came to a grinding halt.  It wasn't dead, it just wasn't making much forward progress either.  A quick look at the heap stats showed we were nearly at capacity and that garbage collection was eating all CPU time, thus not allowing any real work to happen.  Looks like it is time to whip out the ol' memory profiler.  This time I went with Eclipse's Memory



  [Read more...]
Arbitrary Replication/Proxy Topologies with MySQL Sandbox
Employee +0 Vote Up -0Vote Down
How would you like to use MySQL Sandbox to deploy and test any replication topology imaginable? Do you want to describe load balancer and proxy configurations, too? If you already love sandbox (or even if you don't), and you want to play with arbitrarily complex replication/proxy topologies... read on.

Sandbox Is Your Friend

MySQL Sandbox is a great tool. As of this writing, version 3 is feature complete -- and the new features are greatly desired. Replication support has been there a while for master/slave setups with as many slaves as you want and for replication rings of any size. Version 3 adds multi-level replication trees on a multi-sandbox using sbtool.

But what if the provided replication setups do not quite match what you need?





  [Read more...]
Database Sharding at Netlog, with MySQL and PHP
+0 Vote Up -0Vote Down

This article accompanies the slides from a presentation on database sharding. Sharding is a technique used for horizontal scaling of databases we are using at Netlog. If you’re interested in high performance, scalability, MySQL, php, caching, partitioning, Sphinx, federation or Netlog, read on …

This presentation was given at the second day of FOSDEM 2009 in Brussels. FOSDEM is an annual conference on open source software with about 5000 hackers. I was invited by Kris Buytaert and Lenz Grimmer to give a talk in the MySQL Dev

  [Read more...]
Database sharding at Netlog (FOSDEM talk slides)
+0 Vote Up -0Vote Down

Here are the slides from yesterday’s presentation about horizontal database scaling through sharding at the mySQL dev room at FOSDEM 2009.

I’ve got a ton of notes and remarks to these slides, which will become available here soon.

Artem’s Top 10 Tech Predictions And Ideas For 2009 And Beyond
+0 Vote Up -0Vote Down

Everyone and their mother are throwing out their predictions for 2009 nowadays, it’s a new fad. It’s like you’re not cool anymore if you don’t have twitter, a Mac, and a set of random predictions for the next 12 joyous months.

So I decided to throw in a few ideas of my own to be part of the cool crowd again (how much cooler can I be already, you might think, and I wouldn’t blame you).

 

Disclaimer (read it, tough guy)

What this post is:

  • about the future of technology and the Internet, 2009 and beyond.
  • my ideas on what is going to happen or should happen. If they happen to match someone else’s ideas – it doesn’t mean I ripped them
  [Read more...]
Showing entries 1 to 30 of 54 Next 24 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.