This probably won't be a very well written post since I am
working frantically on a proof of concept using Pentaho Data
Integration / Kettle for the etl in a new data warehouse project.
I have just a couple days to get it to work or I'll end up having
to use perl...which will make me hurl.
I want to use a mysql stored procedure for the transformation
input, which is easy to do with the "Table Input" step ( just
CALL the stored proc in the SQL section), but the field names of
the result set don't show up downstream in subsequent steps. When
I right click on a downstream step and select "show input
fields", an "I Can't find any fields" messagebox pops up.
Some may find this a minor annoyance, but it makes subsequent
steps difficult to deal with if you cant visualize the structure
of the data stream in the transformation.
I saw some posts recommending the use of a "Select Values" Step,
but for some reason, I …
In one of my previous posts I was writing about how to read other
processes memory [1]. As an example I tried to get the value of
the hard coded MySQL internal InnoDB variable spin_wait_delay
(srv_spin_wait_delay).
In this example we were using gdb or the operating system ptrace
function to retrieve this value. This method has the disadvantage
that it is pretty invasive.
When I was working on a customer support case I had the idea to
solve this by the much less invasive method of User-Defined
Functions (UDF).
UDF were introduced in MySQL 5.0 [2]. They provide the feasibility to enlarge the
MySQL functionality by adding external code.
The clue is now that you also can use this external code to do
some …
One of the cooler things has been worked on in libmemcached lately is the ability to tinkertoy up Memcached servers.
What does that mean?
There are a lot of forks of memcached. Some commercial, some open
source, some that are in house at a variety of companies. They
all share a few common concepts though.
Protocol
Scale Out
The recognition of this lead me to want to create a set of
libraries that anyone, could use to build these types of servers
quickly. After talking to a number of vendors and users it became
pretty apparent that something like this needed to be done. The
goal is to make it easy for people to write protocol compatible
servers in a way that they can focus on the …
I had trouble today on a client site using my MySQL power tools Maatkit and Statpack.
$ ~/scripts/statpack.py --files=mysql.status.091015.080001.txt,mysql.status.091015.090001.txt
Traceback (most recent call last):
File "/home/rbradfor/scripts/statpack.py", line 563, in ?
main()
File "/home/rbradfor/scripts/statpack.py", line 527, in main
locale.setlocale(locale.LC_NUMERIC, '')
File "/usr/lib64/python2.4/locale.py", line 381, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting
$ cat /var/log/slow-query.log | ./mk-query-digest
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "e_US"
are supported and installed on your system.
perl: warning: Falling back to the standard …[Read more]
Installing Apache2 With PHP5 And MySQL Support On OpenSUSE 11.1 (LAMP)
LAMP is short for Linux, Apache, MySQL, PHP. This tutorial shows how you can install an Apache2 webserver on an OpenSUSE 11.1 server with PHP5 support (mod_php) and MySQL support.
A client recently asked me to fix some Cacti graphs that had broken after upgrading the Cacti templates I wrote for MySQL. The symptoms were weird; I’m not sure I understand fully what happened, but some of the graphs were OK and some had only part of the data they were supposed to. Some graphs would have one data element as usual, and others would be nan (not a number).
After turning on the debug logs, I found that the script was returning the data correctly — it was not a script problem. But after Cacti got the data from the script, it wasn’t associating it correctly with the RRD archives. Here’s a log message:
10/14/2009 12:05:05 PM - CMDPHP: Poller[0] Host[11] DS[1270] CMD: /usr/bin/php -q /opt/cacti/scripts/ss_get_mysql_stats.php --host dbserver --items bj,bm --user --pass , output: bj:68 bm:64 10/14/2009 12:05:05 PM - CMDPHP: Poller[0] DEVEL: …[Read more]
Here’s a quick flowchart for the different ways that Kontrollbase gathers and processes data.
|
|
The Spider storage engine should be already known to the community. Its version 2.5 has recently been released, with new features, the most important of which is that you can execute remote SQL statements in the backend servers. The method is quite simple. Together with Spider, you also get an UDF that executes SQL code in a remote server. You send a query with parameters saying how to connect to the server, and check the result (1 for success, 0 for failure). If the SQL involves a SELECT, the result can be sent to a temporary table. Simple and effective. |
In addition to the Spider engine, Kentoku SHIBA has also created
the …
OpenSQL Camp Portland 2009 is coming up on the 14th and 15th of November. Eric Day (of the Drizzle project) is the lead organiser this time around.
I went to the first edition in Charlottesville VA last year which was organised by Baron Schwartz (Percona). It was a great event, like other unconferences but with specific focus on database technologies. Monty (MySQL), Brian (Drizzle), Richard (SQLite), Jim (Interbase/Firebird/Falcon), Bruce (PostgreSQL) were all these, as were various storage engine builders. Very interesting, and lots of informal fun. If you’re anywhere near, do go!
Even though noone from our gang is able to make it to this one, Open Query is sponsoring this event – for all the above reasons. It rocks and deserves every support.
At Days of Wonder we are huge fans of MySQL (and since about a year of the various Open Query, Percona, Google or other community patches), up to the point we’re using MySQL for about everything in production.
But since we moved to 5.0, back 3 years ago our production databases which hold our website and online game systems has a unique issue: the mysqld process uses more and more RAM, up to the point where the kernel OOM decide to kill the process.
You’d certainly think we are complete morons because we didn’t do anything in the last 3 years to fix the issue
Unfortunately, I never couldn’t …
[Read more]