FTP adapter -- before get ftp raw command: Quote site trim 0

[lindaBztk]

Hi All,

I am trying to get rid of the end-of-line trimming when getting a file
from IBM 400 FTP server through BizTalk native FTP adapter.

A way to do that is to execute: Quote site trim 0 before get. I can have
the expected result through windows command prompt window. However, after
putting the "Quote site trim 0" in the "Before Get" in biztalk ftp receive
location, I got run-time error:
There was a failure executing the receive pipeline:
"Microsoft.BizTalk.DefaultPipelines.PassThruReceive,
Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35" Source: "Pipeline " Receive Port:
"Receive_Download" URI: "ftp://ServerName:21/Test2.*" Reason: The command
"Quote SITE trim 0" failed on the FTP server.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

[Stefan]

Linda,

Could you please log the FTP communication in BizTalk and post us the
list of commands that BizTalk Server is sending to your FTP Server.

In the FTP ReceiveLocation you can specify where the log is to be
saved

Regards,
Stefan
http://www.itsconsulting.fr
http://www.dzair.com

[lindaBztk]

It looks the get command wasn't executed from the log file. I remember Sam
Ritchie mentioned the similar problem:
http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=3481029&SiteID=17

But the hotfix on BizTalk 2006 is nowhere mentioned in R2. He was able to
work around in an odd way -- by setting a temp folder combined with Binary
transfer mode, which to me is not an option. (if I use “Binary” , I would get
a bunch of invisible characters in the file cause the file comes from AS400).

< 220-QTCP at XX.XXXXXX.com.
< 220 Connection will close if idle more than 5 minutes.
> USER user1
< 331 Enter password.
> PASS xxxx
< 230 user1logged on.
> PWD
< 257 "USER1" is current library.
> SYST
< 215 OS/400 is the remote operating system. The TCP/IP version is "V5R3M0".
> PWD
< 257 "USER1" is current library.
> TYPE A
< 200 Representation type is ASCII nonprint.
> PORT XX,XX,XX,XX,XX,XXX
< 200 PORT subcommand request successful.
> LIST Test5.*
< 125 List started.
< ORCOM *MEM TEST5.TXT
< 250 List completed.
> SITE trim 0
< 250 TRIM trailing blanks when sending database files option set to 0.
> PWD
< 257 "USER1" is current library.

[Stefan]

Linda,

This is wired, I can see no error in the log you provided, so
everything went supposedly fine?
At first glance I don't think it is an FTP issue...

1- In your Pipeline Error, it is a passthrough, could you create a
send port and filter it to receive data from your receive port and see
if there is any file sent by your send port?

2- can you extract the file from the BizTalk Administration Console
can you see it's content?

[lindaBztk]

There is no error, but the file is not transferred either. so there is
nothing returned from FTP receive location.

[Stefan]

Linda,

I'm just trying to help.

Look you say : "I am trying to get rid of the end-of-line trimming
when getting a file from ... "
Why are you trying to get rid of the end-of-line trimming, why don't
you try to include it in your Schema? or am I completely lost with
your descriptions ? :-)

[lindaBztk]

Oh, I guess i didn't give enough background of this. The file biztalk gets
from AS400 FTP, is a flat file with a lengh of 128 characters at each line.
After this file is GET from AS400 ftp server by BizTalk FTP adapter with
ASCII mode or by windows FTP command,all blanks after the last character of
each line are trimmed. (Frankly, I am not sure why this is happening. I guess
it's caused by the discrepancies between windows and mainframe).

You can imagine this trimmed version flat file cannot work with any flat
file schema, cause you never know how many blank spaces are gonna be trimmed
at each line.

In Windows FTP command prompt windows, I could prevent that from
happening by issuing a FTP script: "Quote Site Trim 0" just before the Get
command. All I need is to put this "Site Trim 0" in the FTP adapter's "Before
Get". Unfortunately, it didn't work out as expected.

That's the story. I've been trying to work around this issue through a
thirty party FTP adpater -- nsoftware FTPS. But currently got no luck to GET
files from a AS400 ftp server, I wonder if it supports connectivity to AS400.

Thanks.

Linda

[spencer...@gmail.com]

Hi Linda,

I work in the tech support department here at nsoftware and saw this
post. Which version of our BizTalk adapters are you using? If you're
using V2 (our current version) you should see a few properties that
allow you to send raw FTP commands to the FTP server at various
stages. The properties are After Connect, Before Get, and After Get.
It sounds like you may have already tried setting the Before Get
property of our adapter to the raw command "Quote Site Trim 0" that
you wanted to send but it still wasn't working?

I'm also not clear if you're running into an actual error here with
our adapter, or if you're seeing that the file line endings are not
preserved. Can you clarify that please?

As for the transfer mode and the files getting trimmed, there is also
a Transfer Mode property of the adapter that can be set to Binary.
Have you tried this out?

Please also feel free to e-mail us at sup...@nsoftware.com so we can
have a quicker turnaround time on answering you.

- Spencer
/n software

[Stefan]

Linda,

You have an option in your flat schema allowing early termination, so
I think you don't need to care about the trimming stuff (only if it
occurs at the end of the line)

Would you try this solution

[lindaBztk]

Hi Spencer,

Thanks very much for the reply. The version we are using is V2.

First of all, as you suggested, I tried nsoftware FTP adapter connecting
to a windows FTP server with a raw FTP command in "Before Get", the problem
seems to be gone. -- Great!!! At least somebody's got an alternative solution
to work around this MS FTP Adapter bug.

However, Getting files from AS400 FTP server is what I really need. So
far I haven't been able to make nsoftware FTP work with AS400 FTP server even
with no FTP raw command at all. In fact, I haven't reached the point of
checking trimming spaces in the file yet, because I cannot get the file at
the first place.

To help debugging, I switched the “Trace Mode” to “Debug”, here is what's
logged in the event log(In total 12 entries):

Entry#1:
Adapter: "nsoftware FTP Receive Adapter" (FTP:2.3.2994.0)
URI: "FTP://HM:21/Test666.txt"
Trace Mode: DEBUG
Current thread: 35
Checking machine license.

Entry#2:
[The first 4 lines are the same as previous event entry]
Preparing to download files.

Entry#3:
[The first 4 lines are the same as previous event entry]
Setting up connection to FTP server.

Entry#4:
[The first 4 lines are the same as previous event entry]
Logging on to FTP server.

Entry#5:
[The first 4 lines are the same as previous event entry]
Executing AFTER CONNECT script. (Although I don’t have anything set for
“After Connect”)

Entry#6:
[The first 4 lines are the same as previous event entry]
AFTER CONNECT script complete.

Entry#7:
[The first 4 lines are the same as previous event entry]
Setting FTP server mode to passive.

Entry#8:
[The first 4 lines are the same as previous event entry]
Changing remote directory on FTP server.

Entry#9:
[The first 4 lines are the same as previous event entry]
Changing server mode to Binary.

Entry#10:
[The first 4 lines are the same as previous event entry]
Listing directory.

Entry#11:
[The first 4 lines are the same as previous event entry]
Logging off FTP server. (Note: This is really not supposed to happen, the
Text666.txt file is there, the Adapter is expected to transfer the file
instead.)

Entry#12:
[The first 4 lines are the same as previous event entry]
Operation complete.

Eventually, there is no file transferred. Please help!

Attached is the adapter configuration:
<AdapterConfig vt="8">
<Config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Account />
<AfterConnect />
<AfterGet />
<BeforeGet />
<DeleteMode>0</DeleteMode>
<FileMask>Test666.txt</FileMask>
<FirewallHost />
<FirewallPassword>******</FirewallPassword>
<FirewallPort>0</FirewallPort>
<FirewallType>0</FirewallType>
<FirewallUser />
<FTPPort>21</FTPPort>
<FTPServer>HM</FTPServer>
<LocalFile />
<Overwrite>False</Overwrite>
<Passive>True</Passive>
<Password>******</Password>
<RemotePath />
<SSLAcceptServerCert />
<SSLCert />
<SSLStartMode>3</SSLStartMode>
<TempDir>d:\temp</TempDir>
<TransferMode>2</TransferMode>
<ErrorThreshold>100</ErrorThreshold>
<FileCount>0</FileCount>
<FileSize>0</FileSize>
<Other />
<PollingInterval>60</PollingInterval>
<SuspendOnError>False</SuspendOnError>
<Timeout>60</Timeout>
<TraceMode>0</TraceMode>
<User>biztalk</User>
<uri>FTP://HM:21/Test666.txt</uri>
</Config>
</AdapterConfig>

Thanks a lot.

Linda

[lindaBztk]

Hi Spencer,

Thank you very much for your help on this.

Exactly like you said, nsoftware FTP adapter is expecting a standardized
directory listing format but the AS400 machine is returning something more
atypical so the adapter can't parse what files are there and what aren't.
After I followed your instruction to add "UseSimpleDirList=true" in
Tuning->Other property, the file is received successfully. Then after I put
"SITE Trim 0" in "Before Get", file comes back with no trimming problem at
all, which is what I've been fighting for days, you saved my day.

Thanks a lot.

Linda