In a heavily webbed, automated, interconnected world with most data stored on relational databases, we can sometimes forget that there are indeed many situation where you simply want to FTP a file from one place to another.
That process in itself holds many dangers as I pointed out to someone on the forum today. Let me re-cap that post here on the blog…
Suppose your files are coming in using FTP to a local directory.
A file is being written, let’s call it
FILE_20070328.txt.
Now, in advance you don’t know the size of that file. Let’s say
it’s 10MB and takes 30 seconds to FTP.
In your transformation you detect this file and start working.
Chances are very high that you’ll be reading an incomplete
file. (See also this
technical tip on variables and file handling)
There are 2 ways to …
[Read more]