Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский
Displaying stored procedure result set meta data in Pentaho
+0 Vote Up -0 Vote Down
This probably won't be a very well written post since I am working frantically on a proof of concept using Pentaho Data Integration / Kettle for the etl in a new data warehouse project. I have just a couple days to get it to work or I'll end up having to use perl...which will make me hurl.

I want to use a mysql stored procedure for the transformation input, which is easy to do with the "Table Input" step ( just CALL the stored proc in the SQL section), but the field names of the result set don't show up downstream in subsequent steps. When I right click on a downstream step and select "show input fields", an "I Can't find any fields" messagebox pops up.

Some may find this a minor annoyance, but it makes subsequent steps difficult to deal with if you cant visualize the structure of the data stream in the transformation.

I saw some posts recommending the use of a "Select Values" Step, but for some reason, I could not get that to work. I spent way too much time trying several different permutations in this step.

This can't really consider this a bug since it seems unreasonable to expect software to reverse engineer a stored procedure to extract result set metadata.

RESOLUTION:

My work around seems to be working ok so far:

1) Create a "Generate Rows" Step, with 0 ( zero) rows output. In this step, configure the fields and datatypes to match the result set of your stored procedure. This step is essentially nothing more than a specification.

2) Create a "Table Input" step that calls your stored procedure.

3 ) Create an "Append Streams" step that merges 1) and 2) above. Set the Head Hop to 1) above, and the Tail Hop to the step from 2) above.

4) create your downstream step and create a hop to it from the "Append Streams". If you right click on this step and select "show input field fields", you can now see the structure of the stream.

Here is a rough picture:








Votes:

You must be logged in with a MySQL.com account to vote on Planet MySQL entries. More information on PlanetMySQL voting.

Planet MySQL © 1995-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Sun Microsystems, Inc. and does not
necessarily represent the opinion of Sun Microsystem, Inc. or any other party.