Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Previous 30 Newer Entries Showing entries 61 to 89

Displaying posts with tag: Main (reset)

Database I/O and Schedulers
+0 Vote Up -0Vote Down

There has been some great discussion lately about internal database schedulers and I/O handling on the Drizzle mailing list, and I thought I’d blog about it to summarize and see what others thought. These are ideas we’re toying around with for Drizzle, but they could be interesting for other databases (or other types of servers). The key things we are focusing on are modularity and concurrency. We also need well defined interfaces to allow anyone to come along and write/rewrite a component for their own needs.

History

First, a little background for those not familiar with Drizzle or MySQL internals. In MySQL, a client is accepted, assigned to a THD (thread) object, and is given an actual thread to run in. The client basically sits in a loop of reading a command, parsing it, executing it, and writing the result back. This

  [Read more...]
libdrizzle 0.2.0 now in Drizzle
+0 Vote Up -0Vote Down

I’ve spent the past couple weeks ripping out the old client library (based on libmysql) from the Drizzle client utilities (drizzle, drizzledump, drizzleadmin, …). I’ve replaced it with new code that uses the new libdrizzle. Brian merged this patch into the trunk yesterday, so we are one step closer to being rid of the old library! It is still used internally within the server, so my next task is ripping it out from there.

This means that the old library is no longer installed, so all new client and application development needs to use the new libdrizzle. It also means libdrizzle is now required to be able to build Drizzle. While I’ve done my best to convert the old utilities, I’m sure I’ve missed a few things. Please report any bugs you find with

  [Read more...]
The Future of Databases
+0 Vote Up -0Vote Down

Reading “Test Center: Slacker databases break all the old rules” has finally inspired me to write this post, which I’ve been meaning to do for a while. The title of my post may be bold, but it is just one version of how things may go moving forward (and was meant to grab your attention). :)

First off, a few issues I have with the article mentioned above:

  • “…tacking the two letters “db” onto a pile of code that breaks with the traditional relational model.” - Since when does a database need to be relational? Sure, relational databases are the most popular, but they are only a subset of all the possible database types (let’s not forget hierarchical or tuple stores).
  • “None of them is right for everyone, and
  [Read more...]
Drizzle Developer Day - 10 spots left!
+0 Vote Up -0Vote Down
UPDATE developer_day_signup
SET name=YOU!
WHERE (attending_mysql_uc=True OR attending_percona_conference=True)
      AND shape_future_database=True;
Goodbye Concentric, Hello Drizzle
+0 Vote Up -0Vote Down

This Sunday I’ll be flying down to the bay area for my final week of work at Concentric. The following week I’ll be starting as a full-time Drizzle/Gearman developer with Sun Microsystems. This was the position Brian Aker mentioned back in January.

Over nine years ago I was flying to the bay area for my job interview at Concentric. It’s been a great run and I’ve learned a tremendous amount there, writing everything from customized, multi-threaded HTTP and DNS servers to a clustered, hierarchical object database with a full query language. The environment at Concentric feels like a family, probably because a number of co-workers have been there even longer than me. It was a tough decision to make, and I wish them all the

  [Read more...]
Gearman + MySQL Webinar Slides
+0 Vote Up -0Vote Down

Thanks to those of you who came to listen in on the webinar. I’ve posted the slides here if you want them. If you have any follow-up questions, check out #gearman on irc.freenode.net.

New Open Source Conference
+0 Vote Up -0Vote Down

Living in Portland, OR, I was sad to see O’Reilly move OSCON to San Jose this year (don’t they have enough with MySQL, Velocity, …?). So what did the Portland locals do? Start their own:

Open Source Bridge (we have a lot of bridges here in Portland).

It looks like they just opened up their call for papers recently, quoting from their page: “Open Source Bridge is accepting proposals now through March 31st for our 2009 conference, which will take place June 17–19 at the Oregon Convention Center in Portland, OR.”

MySQL + Gearman Webinar
+0 Vote Up -0Vote Down

Tomorrow I’m going to be giving a free webinar on Gearman through MySQL (http://mysql.com/news-and-events/web-seminars/display-280.html). I’ll be describing the basics of Gearman and then look at a few common use cases. I’ll even be touching on some more advanced topics such as Map/Reduce and persistent queues.

Click here to sign up now! (http://mysql.com/news-and-events/web-seminars/display-280.html)

libdrizzle + SQLite hack
+1 Vote Up -0Vote Down

I’ve been wanting to try something with libdrizzle since I added the server protocol interface. After an hour or two of hacking, I came up with:

lap> sqlite3 farm
SQLite version 3.5.9
sqlite> CREATE TABLE animals (
   ...>   name  VARCHAR(255) NOT NULL,
   ...>   sound VARCHAR(255));
sqlite> INSERT INTO animals VALUES ("Cow", "Moo");
sqlite> SELECT * FROM animals;
Cow|Moo
sqlite>

Next, using a new program I hacked together in the libdrizzle examples directory:

./sqlite_server -m -v farm 54321

Now, in another terminal:

lap> mysql -h 127.0.0.1 -u root -P 54321
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: libdrizzle+SQLite

mysql> INSERT INTO animals VALUES ("Sheep", "Baa");
Query OK, 0 rows affected (0.14 sec)

mysql> SELECT * FROM
  [Read more...]
Drizzle PHP Extension 0.1 Released!
+0 Vote Up -0Vote Down

Building on top of my last post, the PHP extension for the new client and protocol library is ready as well! You can download the tarball on the Launchpad download page. The PHP extension exposes the same set of client functions that the C library provides currently, and is mostly just a wrapper to provide the PHP specific handling. To install, you’ll need the PHP development packages installed (if you have ‘phpize’ command you’re all set). After extracting the tarball, just run:

phpize
./configure
make
make install

And then add the following line to your php.ini:

extension=”drizzle.so”

I realize there is no documentation at this point, we’re working on that (please get in




  [Read more...]
Drizzle Client & Protocol Library 0.1 Released!
+0 Vote Up -0Vote Down

I’m happy to announce the first release of the new libdrizzle. This is a complete rewrite of the protocol and client interface for the Drizzle project and provides new functionality that the old library based on MySQL code did not offer. Oh, and this new library supports both Drizzle and MySQL protocols, so you can use it with MySQL servers as well!

So, what’s different?

  • BSD License.
  • Complete non-blocking I/O support.
  • Concurrent query interface, letting you run multiple queries at the same time on any number of servers.
  • Improved buffering support. Instead of just result and row buffering, you can also buffer per field, or not at all (reading fields in chunks). This can be very useful for streaming large blobs through your client.
  • Complete server protocol
  [Read more...]
Gearman Server & Library 0.3 Released, Updated UDFs
+0 Vote Up -0Vote Down

This new release of Gearman includes improved client library code, a few bug fixes, new benchmarking tools, and man pages for the API. I also changed the default port in this release to use our new IANA assigned port, 4730 (was previously 7003). This means if you try to mix existing APIs or job servers with this release, you’ll need to specify port options on one side to make sure they match up. You can find the new tarballs on the Gearman Download page or on Launchpad.

I’ve also released a new version of the Gearman MySQL UDFs due to some of the C library changes. You can find the updated tarballs on the Gearman Download page or on

  [Read more...]
Gearman has a port!
+0 Vote Up -0Vote Down

Over a month ago I applied for a real port for Gearman (before the default port was conflicting with AFS). After some back and forth with IANA:

We have assigned the following registered port number to 'gearman'
with you as the point of contact:

gearman         4730/tcp   Gearman Job Queue System
gearman         4730/udp   Gearman Job Queue System

See: http://www.iana.org/assignments/port-numbers

I’ll be updating this in the C version for the next release (which will trickle down into the MySQL UDFs). I’m also in the process of notifying other API and server maintainers to get their ports updated. So, if you use Gearman, watch out for possible port mismatches while the various packages transition!

Gearman Server & Library 0.2 Released
+0 Vote Up -0Vote Down

This release includes a rework of the Gearman worker library internals, along with some improvements and bug fixes to the server, gearmand. You can find it off the wiki download page (http://www.gearmanproject.org/) or directly from Launchpad.

Remember to check out the various APIs, including the new MySQL UDFs released last week!

Gearman Persistent Queues & Replication
+0 Vote Up -0Vote Down

Now that the core of Gearman is rewritten in C, I’ve started working on a new module to provide persistent queues and queue replication. The current Gearman implementation is fine when the jobs can be unreliable, but there are many cases when we want to guarantee that jobs get run.

Because Gearman is such a flexible system, there are a couple of places where we can plug in a persistent queue and/or replication. Now, in most cases, these features are only going to be useful for background jobs since failed foreground jobs can be detected and restarted by a client. Many applications using Gearman want to just throw a job in the queue and go back to doing something else, not caring when it gets done. Enter persistent queues. It may also be the case that a job should be run multiple times, mot likely on a separate set of servers for backup and geographic redundancy.

  [Read more...]
Gearman MySQL UDFs: URL Processing
+0 Vote Up -0Vote Down

Yesterday we announced the new release of the Gearman server, library, and UDFs, and now I’d like to provide a more useful example. This example assumes you’ve taken the steps to install the C server and MySQL UDFs as explained in that post, and now we’ll be using those pieces and the Perl API to show how to do URL processing. This could be used by a number of applications - anything where you have a URL, need to fetch it, possibly do some processing, and then store it somewhere (RSS feed cache, pulling links out of a HTML page, image conversion, …). In this example, MySQL is being used as the repository and “trigger point”, helping track the requests and results. Since we’ll be using Perl for our worker code, the first thing to do is install the Perl API from CPAN:

> perl -MCPAN -eshell
cpan shell -- CPAN exploration and
  [Read more...]
New Gearman Server & Library in C, MySQL UDFs
+0 Vote Up -0Vote Down

We’re happy to announce the first version of the Gearman rewrite in C, along with some interesting new MySQL UDFs based on the C library. Check out the Gearman wiki for an overview of what this is, download details, and API documentation.

For the anxious, here is a quick how-to:

Download: http://launchpad.net/gearmand/trunk/0.1/+download/gearmand-0.1.tar.gz
tar xzf gearmand-0.1.tar.gz
cd gearmand-0.1/
./configure
make
make install

You should now have the job server (gearmand) installed in /usr/local/bin, along with headers in libraries in /usr/local/lib & include. You can now see it work by running some simple clients and workers in the examples/ directory:

gearmand &      (this is assuming /usr/local/bin is in your path)
cd examples
./reverse_worker &
./reverse_client "Hello, Gearman!"

If everything went well,

  [Read more...]
Faster Gearman
+0 Vote Up -0Vote Down

The gearmand job server written in Perl (current production server from Danga):

mysql> SELECT length(gman_do("reverse", repeat('x',10000000))) AS test;
+----------+
| test     |
+----------+
| 10000000 |
+----------+
1 row in set (49.08 sec)

The new gearmand job server written in C:

mysql> SELECT length(gman_do("reverse", repeat('x',10000000))) AS test;
+----------+
| test     |
+----------+
| 10000000 |
+----------+
1 row in set (0.30 sec)

Mmm, efficiency. Oh, and are those some new MySQL UDFs? Much more coming soon…

UC 2009 Proposals
+0 Vote Up -0Vote Down

Like a few others I’ve seen this week, I had two proposals for the MySQL Conference & Expo 2009 accepted. I’m very excited for both topics (for different reasons), and will be blogging about each of them in more detail soon. They are:

  • libdrizzle: A new client library for Drizzle and MySQL - This will be a detailed introduction of the new library, covering important topics like client-side buffering, non-blocking I/O, and concurrent queries. If time permits I’ll also go over features in the new Drizzle protocol.
  • Map/Reduce and Queues for MySQL using Gearman - I’ll be co-presenting this one with Brian Aker, and we will show how the work we’ve been doing with Gearman can be tied into MySQL (and Drizzle).
Next Project
+0 Vote Up -0Vote Down

I spent this past week down in San Jose, CA at my employer’s office for team meetings and to officially kick-off my next big project. The design and architecture was very well received, and I drummed up some excitement with Gearman and working with the OSS community in general (which we’ve not done too much of in the past). We’ll be developing it entirely on Launchpad under GPLv2, and I’ll be writing a number of blog posts covering each component in detail. Why would anyone else find this interesting? It covers many topics of how to write a high-performance application in the cloud. Specific topics will include Gearman, persistent Gearman queues, eventual consistency data models (and related schemas), lightweight Map/Reduce for real-time applications, and how to combine all this with MySQL

  [Read more...]
Open SQL Camp
+0 Vote Up -0Vote Down

Last weekend I attended the OpenSQL Camp in Charlottesville, VA. There was a great turnout, and Baron did an excellent job organizing it! I saw a few folks I met at OSCON over the summer, along with meeting many new people. What a great group - intelligent, fun, and know how to get things done. I had some great conversations, especially with Brian, Stewart, Arjen, Patrick, Mark, and Jay. The food was great too, I was a bit worried about finding vegan food there. Oh, and there was the wine bar, and my new found love for dessert wine.

  [Read more...]
No Spam! Sorta…
+0 Vote Up -0Vote Down

So, I installed one of those math question anti-spam plugins for WordPress comments. It stopped the spam! Unfortunately it also stopped all valid comments when the answer was right, although it did work when I first set it up. So if you tried leaving a comment (I see a few were denied) for one of the recent Drizzle or MySQL related posts (like the New libdrizzle), please try again. :)

Anyone have a suggestion for a good anti-spam plugin for WordPress comments? I’ve looked a bit but nothing really stands out.

New libdrizzle
+0 Vote Up -0Vote Down

What’s the new libdrizzle? It’s a complete rewrite of the client library for the Drizzle project, but it also has full support for the current MySQL protocol (4.1+). Right now Drizzle uses the same protocol as MySQL, but work is being done to design a new, more robust protocol. Even when the Drizzle protocol changes, I plan to keep full support for the MySQL protocol since there is a need for a good low-level non-blocking client library. Also, once libdrizzle turns into a full protocol library (server and client packets) it could make for some interesting Drizzle/MySQL hybrid proxies.

Over the weekend I made a lot of progress on the new library, and last night I just finished up the first pass at the core functionality. It can now do full query and result processing. Currently only non-cached results are being used (no store_result() function), but the cached

  [Read more...]
Back on Track
+0 Vote Up -0Vote Down

After an amazing summer of getting married, graduating college, adopting a new dog, and taking a much needed extended vacation, I’m now back on track with where I left off. I’ve been making good progress on the asynchronous MySQL library I talked about in this post in the form of a new drizzle client library. This library is also compatible with MySQL since they share the same protocol, and if drizzle changes in the future I plan on supporting current and new MySQL protocols as well. All of the connection and I/O overhead is mostly done, and I’m just working through the protocol bits now. I’m hoping to have something ready to show and talk about for the OpenSQL camp.

In other news, I’ve recently started working with the

  [Read more...]
The Big Mac Dilemma
+0 Vote Up -0Vote Down

Last week I was at OSCON (more on that later) and I was surrounded by a mix of laptops - most of them running Ubuntu or OS X. This caused an internal dilemma I’ve been having for years to resurface: is using a Mac “selling out” from your OSS ideals? I still use a PowerBook and last fall I purchased an iMac (guilt included). Previously I had been running Slackware Linux on a wide range of desktops and laptops (this was since 1995). I switched because I was tired of dealing with all those little pesky hardware compatibility issues, especially on laptops, and I thought the time saved there would be a good tradeoff for new development. Well, I don’t think I can use that excuse anymore. Ubuntu is pretty slick and takes many of those issues away. A few laptop manufacturers are even shipping with Linux now. There is even an argument using OS X for development is more difficult now (try compiling large

  [Read more...]
Looking for MySQL DBA
+0 Vote Up -0Vote Down

Concentric, the company I work for, is looking for a MySQL DBA! The position is in San Jose, CA, and here is a link to the official job posting. To apply, please email your resume and a cover letter to SELECT REVERSE(’moc.ox.ws@semuser’);. Include “MySQL DBA” in your subject line. Here is the description:

Concentric, an XO Communications Service, is looking to grow its Engineering team with a MySQL Database Administrator. The successful candidate will have great technical depth and breadth to span our applications, server and web service. We want to talk to people who can think out-of-the-box to help us find unique solutions in a fast moving environment.

Responsibilities

  • Design and support the MySQL 5 databases for customer-facing and back-end systems used in a high volume internet services
  [Read more...]
OSCON
+0 Vote Up -0Vote Down

I’m getting geared up for OSCON next week. It looks like there will be quite a few regular sessions, along with a number of Birds of a Feather sessions, focused on MySQL.

For those making it out, I thought I would mention a couple fun things to do around Portland. I’ve been living here about three years now, so I haven’t seen it all, but these are some must-sees.

  • Powells Technical Books - I know they are a conference sponsor and will have a booth, but try to make it over to the real store (it’s just a short walk across the bridge from the convention center). Not only is it one of the best technical book
  [Read more...]
Hot off the press
+0 Vote Up -0Vote Down

The UPS decided to visit today and left this on the front doorstep:

Woot! If it’s even half as good as the tips and tricks described in the authors’ blogs we should all be in for a treat.

…and while I’m posting pictures, I can’t resist sharing this photo we took while visiting my family in Maine last week:

Yes, that’s a full size riding lawn mower strapped to the top of a small Ford Escort hatchback.

Summer Reading
+0 Vote Up -0Vote Down

With the bulk of my classes over, I finally have time to read some hand-picked books (rather than those 1000 page dry volumes required for courses). This is perfect timing since the new “High Performance MySQL” book was just released. I got a little carried away with my order, but felt I should catch up on some books I’ve been eyeing for some time. Anyone else care to share their summer reading list?

By June 30th, I should have:

  • High Performance MySQL: Optimization, Backups, Replication, and More
  • The Art of Multiprocessor Programming
  • The Definitive ANTLR Reference: Building Domain-Specific Languages
  • Programming Erlang: Software for a Concurrent World
  • The Joy of Vegan Baking: The Compassionate Cooks’ Traditional Treats and Sinful Sweets

(The last book is for both Wendy and me, it should help provide tasty treats while reading

  [Read more...]
Previous 30 Newer Entries Showing entries 61 to 89

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.