FTP/SSL from OpenVMS (client) to Unix Filezilla (server) failure

[gopalakrishnan]

OpenVMS Version 8.4
TCP/IP Version V5.7 - ECO 2
FTP/SSL client - OpenVMS as above (my company)
FTP Server - Unix Filezilla (external service provider)
Server command port: 22
(non standard port number dictated by external SP)
Servr data ports 5000-5099
Problem:
Connection is successfully opened, but data transfers
fails

OpenVMS DCL commannd and responses below:

$ FTP/SSL server 22 /user=X /passw=Y

220-Welcome to the SFTP service.
220-
Connected to (server)
234 Using authentication type TLS
200 PBSZ=0
200 Protection level set to P
331 Password required for z
230 Logged on

FTP> cd /upload
250 CWD successful. "/upload" is current directory.

FTP> cd /download
250 CWD successful. "/download" is current directory.

FTP> cd /NoSuchFolder
550 CWD failed. "/NoSuchFolder": directory not found.

FTP> quote pasv
227 Entering Passive Mode (nnn,nnn,34,121,19,220)

FTP> enable log
Bell off.
Debugging on (debug=1).

(all the above work,
but the following fails
..and also a "get" of a file I know exists in one of the folders)

FTP> dir
---> PORT 10,1,133,79,192,45
200 Port command successful
---> LIST
150 Opening data channel for directory list.
%TCPIP-E-FTP_DATACONF, cannot establish data connection with remote host
-SYSTEM-F-CONNECFAIL, connect to network object timed-out or failed
425 Can't open data connection.

(10.1.133.79 is the company-internal IP of the OpenVMS virtual machine and not of the firewall/ISA)


--http://compgroups.net/comp.os.vms/

[Stephen Hoffman]

On 2012-08-05 20:20:07 +0000, gopalakrishnan said:

> OpenVMS Version 8.4
> TCP/IP Version V5.7 - ECO 2
> FTP/SSL client - OpenVMS as above (my company)
> FTP Server - Unix Filezilla (external service provider)
> Server command port: 22
> (non standard port number dictated by external SP)
> Servr data ports 5000-5099
> Problem:
> Connection is successfully opened, but data transfers
> fails
>
> OpenVMS DCL commannd and responses below:
>
> $ FTP/SSL server 22 /user=X /passw=Y
>
> 220-Welcome to the SFTP service.

Nonstandard TCP port or not, TCP 22 is the standard port for ssh (and
sftp), and not FTP/SSL.

And you're connecting to an sftp server, based on the 220 response shown.

Which means an attempt to use FTP/SSL (aka FTPS) here is likely not
appropriate.

Try the connection again, though with sftp this time.



--
Pure Personal Opinion | HoffmanLabs LLC

[Steven Schweda]

And, even if you did have the right server to go with your
client, you appear to be operating it incorrectly.

> FTP> quote pasv
> 227 Entering Passive Mode (nnn,nnn,34,121,19,220)

This tells the FTP server to go into passive mode (which
apparently succeeds), but it does not tell the client to
_use_ passive mode. This might explain the failures which
follow.

If you want passive mode, then tell your client program
that. Do not use "quote" to send a message to the server
behind the client's back.

[gopalakrishnan]

Thank you for the message Steven

Tried that too and it does not work either

There sure is a difference: when I do "set passive on" OpenVMS FTP command, it does not send a port command after I enter the "dir" command. It sends a PASV command instead ....as expected???

Below is the console log:

220-Welcome to the Xxxx SFTP service.
220-
220-Use of this service is subject to the security and acceptable use policies
220-of Xxxx; copies of these documents can be obtained from
220 secu...@xxxx.co.nz for approved usage by Xxxx clientelle.
Connected to SFTP.XXXX.CO.NZ.

234 Using authentication type TLS
200 PBSZ=0
200 Protection level set to P
331 Password required for z
230 Logged on

FTP> enable log
Bell off.
Debugging on (debug=1).

FTP> set passive on
Passive is ON.

FTP> cd /upload
---> CWD /upload

250 CWD successful. "/upload" is current directory.

FTP> dir
---> PASV
227 Entering Passive Mode (117,120,34,121,19,145)
%TCPIP-E-FTP_NETERR, I/O error on network device
-SYSTEM-F-TIMEOUT, device timeout

%TCPIP-E-FTP_DATACONF, cannot establish data connection with remote host

-SYSTEM-F-TIMEOUT, device timeout

[Steven Schweda]

> [...] It sends a PASV command instead ....as expected???

If you have doubts, then you might run the experiment
using some other FTP server, such as the one on the VMS
system itself. (A test without the /SSL might be a good
idea, too. Trusting new features in TCPIP is seldom safe.)

[Ken Fairfield]

On Sunday, August 5, 2012 10:07:02 PM UTC-7, gopalakrishnan wrote:
[...]

> 220-Welcome to the Xxxx SFTP service.

^^^^

> 220-
> 220-Use of this service is subject to the security and acceptable use policies
> 220-of Xxxx; copies of these documents can be obtained from
> 220 secu...@xxxx.co.nz for approved usage by Xxxx clientelle.
> Connected to SFTP.XXXX.CO.NZ.
> 234 Using authentication type TLS
> 200 PBSZ=0
> 200 Protection level set to P
> 331 Password required for z
> 230 Logged on
>
> FTP> enable log

^^^^

> Bell off.
> Debugging on (debug=1).

[...]

Either you missed Hoff's message, or you've ignored it.

You're attempting to use an FTP client to talk to an SFTP
server and it isn't going to work.

Is there a reason you aren't using SFTP from VMS? It works.

-Ken

[gopalakrishnan]

Hi Ken

I submitted a reply to Hoff's message and repeated when I did not find my reply here. And still it is not here (I did not get the usual response "message will be reviewd and posted" in those two instances). Hope this gets loaded.

The server truly is FTP over SSL service (the very first line in the SP's documentation). They use "SFTP" to describe the service as being "secure". The service uses port 22 which is not FTP/SSL standard port but SFTP's. Even the server is named "sftp.xxxx.co.nz"

Moreover I can connect to the sevice, view folders and download files using a Windows FilZilla client connecting to "FTPES://sftp.xxx.co.nz", port 22 (and user-id / password)

I have tried SFTP and it does not work

My infrastructure team is now investigating if this has anything to do with our firewall/isa configuration

Regards -tk


--http://compgroups.net/comp.os.vms/ftp-ssl-from-openvms-client-to-unix-filezilla-ser/1519815

[Dirk Munk]

With the windows version you are using explicit FTPS, or FTPES, and that
may point to the problem. From Wikipedia:

Explicit

In explicit mode (also known as FTPES), an FTPS client must "explicitly
request" security from an FTPS server and then step-up to a mutually
agreed encryption method. If a client does not request security, the
FTPS server can either allow the client to continue in unsecure mode or
refuse/limit the connection.

The mechanism for negotiating authentication and security with FTP was
added under RFC 2228, which included the new FTP command AUTH. While
this RFC does not explicitly define any required security mechanisms,
e.g. SSL or TLS, it does require the FTPS client to challenge the FTPS
server with a mutually known mechanism. If the FTPS client challenges
the FTPS server with an unknown security mechanism, the FTPS server will
respond to the AUTH command with error code 504 (not supported). Clients
may determine which mechanisms are supported by querying the FTPS server
with the FEAT command, although servers are not necessarily required to
be honest in disclosing what levels of security they support. Common
methods of invoking FTPS security included AUTH TLS and AUTH SSL.

In the later RFC 4217, FTPS compliance required that clients always
negotiate using the AUTH TLS method. The RFC also recommended FTPS
servers to accept the draft mechanism AUTH TLS-C.


Th use of port 22 is wrong. For FTPS the normal port numbers are 989 and
990. However some implications use the standard FTP port 21 for FTP and
FTPS.

SFTP uses port 22, but as you know SFTP is something very different from
FTP or FTPS.

[Richard Whalen]

On Aug 7, 8:37 am, Dirk Munk <m...@home.nl> wrote:
> gopalakrishnan wrote:
> > Hi Ken
>
> > I submitted a reply to Hoff's message and repeated when I did not find my reply here. And still it is not here (I did not get the usual response "message will be reviewd and posted" in those two instances). Hope this gets loaded.
>
> > The server truly is FTP over SSL service (the very first line in the SP's documentation). They use "SFTP" to describe the service as being "secure". The service uses port 22 which is not FTP/SSL standard port but SFTP's. Even the server is named "sftp.xxxx.co.nz"
>
> > Moreover I can connect to the sevice, view folders and download files using a Windows FilZilla client connecting to "FTPES://sftp.xxx.co.nz", port 22 (and user-id / password)
>
> > I have tried SFTP and it does not work
>
> > My infrastructure team is now investigating if this has anything to do with our firewall/isa configuration
>
> > Regards -tk
>

> > --http://compgroups.net/comp.os.vms/ftp-ssl-from-openvms-client-to-unix...

I suspect that the problem is a firewall that isn't able to interpret
the response to
the PASV or PORT command and allow the port to be opened. The firewall
can't
interpret the response because the command stream is encrypted. You
need to
cause the FTPS client to send the CCC command to the FTPS server. See
http://h71000.www7.hp.com/doc/84final/tcprn/tcp_rnpro_001.html#ftp_ssl

[Stephen Hoffman]

On 2012-08-07 12:37:13 +0000, Dirk Munk said:

> gopalakrishnan wrote:
>> Hi Ken
>>
>> I submitted a reply to Hoff's message and repeated when I did not find
>> my reply here. And still it is not here (I did not get the usual
>> response "message will be reviewd and posted" in those two instances).
>> Hope this gets loaded.

You're apparently not using usenet, you're probably using one of those
sites that re-hosts usenet content.
Try accessing this usenet newsgroup directly.
You can get a free usenet news account at eternal-september.org, and
news readers are available for all platforms.

>>
>> The server truly is FTP over SSL service (the very first line in the
>> SP's documentation). They use "SFTP" to describe the service as being
>> "secure". The service uses port 22 which is not FTP/SSL standard port
>> but SFTP's. Even the server is named "sftp.xxxx.co.nz"
>>
>> Moreover I can connect to the sevice, view folders and download files
>> using a Windows FilZilla client connecting to "FTPES://sftp.xxx.co.nz",
>> port 22 (and user-id / password)
>>
>> I have tried SFTP and it does not work

"Does not work"? Please explain that. And please see
<http://www.mikeash.com/getting_answers.html> for some background on
this topic.

>>
>> My infrastructure team is now investigating if this has anything to do
>> with our firewall/isa configuration

That would be typical, but that also implies that the FileZilla test
wasn't the same network path as the OpenVMS test.

>>
>> Regards -tk
>>
>>
>> --http://compgroups.net/comp.os.vms/ftp-ssl-from-openvms-client-to-unix-filezilla-ser/1519815
>>


If that's the service you're using to access the comp.os.vms newsgroup,
please use usenet news directly.

ftp is a pile of steaming stench, with a side-helping of skunk-stink.

ftp over ssl (aka ftps) is, well, a bandaid atop a steaming pile of
stench. ftp/ftps on the wrong ports is worse; emphasis on ports. The
steaming pile of stench that is ftp/ftps is inherently incompatible
with modern networks, and with firewalls. In particular, ftp/ftps
opens a second port up in the empheral range, which then usually gets
blocked by an intervening firewall, or means expensive
protocol-sniffing firewalls. (And protocl sniffing is tougher with
ftps, for obvious reasons.)

The vastly better protocol here is sftp, which shares three or four
letters with those other protocols and a basic purpose, but is
otherwise quite different. sftp is expressly designed to operate
(securely) in modern networks. Unlike the steaming pile of ftp/ftps
stench.

Put another way, move to sftp where you can.

If you can't use sftp, then you could shovel the bits over to the
Windows PC box and use FileZilla to transfer the files. (I'm here
presuming that your FileZilla test is using the same network path as
the OpenVMS system that you're testing with. If it's not, then
firewalls are back in play, and we can once again enjoy the wafting
stench of ftp and ftps.)

Various versions of cURL can be used to access ftps servers, too, and
there's a cURL port for OpenVMS. (Whether that port does ssl and thus
ftps, I don't know.) And there are probably some Java ftps clients
around.

Or use a VPN, which would reduce the exposure of ftp and ftps to the
vagaries of the intervening network firewalls.

Now the other wildcard here might be the use of "secure ftp", which is
tunneling ftp (gag) through ssh. That would normally use port 22,
because it's using ssh. Though the use of ftpes via Filezilla would
tend to belie that use, though.

In general, if there's a support contract in place with HP, call the HP
support center and have them sort this out. There have been some bugs
in recent versions of the TCP/IP Services for OpenVMS package, and this
might be another one.

And yes, you could reasonably infer I'm not fond of ftp.

[Jose Baars]

Op woensdag 8 augustus 2012 15:28:38 UTC+2 schreef Stephen Hoffman het volgende:

>
> And yes, you could reasonably infer I'm not fond of ftp.
>

:-). In FTP's defense, most browsers, and curl too, automatically use passive mode to do FTP, getting rid of the server opening a random port.
But why we have FTPS, FTP over SSL (also loosely called FTPS) and more than 2 ways to tell how and what to encrypt is really beyond me.
SSH and SFTP implement an application protocol on top of TCP/IP. That has at least three disadvantages:
To prevent setup of SSH tunnels a company needs expensive protocol sniffing
firewalls :-). The SFTP (or really the SSH) RFC's are extensive and open to (wrong) interpretation, as abundantly demonstrated by dozens of servers and clients, and file transfer performance is sometimes not even half of an HTTPS (to avoid the dreaded F word) download or upload, which can be comparably secure. That also is the disadvantage of HTTPS: it can be made much more
insecure.
I don't think I will see something easy, secure and generally accepted in my lifetime.

[Stephen Hoffman]

On 2012-08-08 21:55:09 +0000, Jose Baars said:

> Op woensdag 8 augustus 2012 15:28:38 UTC+2 schreef Stephen Hoffman het
> volgende:
>
>>
>> And yes, you could reasonably infer I'm not fond of ftp.
>>
>
> :-). In FTP's defense, most browsers, and curl too, automatically use
> passive mode to do FTP, getting rid of the server opening a random port.

Which just mean the server needs the whole of the ftp port range open
(or the whole of the ephemeral range) from the Internet, or a firewall
that sniffs ftp traffic.

And some browsers intentionally block URL-based credentials on ftp transfers.

ftp needs to die.

But then I've mentioned my distaste for the protocol.

> But why we have FTPS, FTP over SSL (also loosely called FTPS) and more
> than 2 ways to tell how and what to encrypt is really beyond me.

Schadenfreude is about the nicest reason I can come up with.

> SSH and SFTP implement an application protocol on top of TCP/IP. That
> has at least three disadvantages:
> To prevent setup of SSH tunnels a company needs expensive protocol
> sniffing firewalls :-).

Peeking in the encrypted traffic generally means a deliberately weak
implementation, a certificate compromise (as has happened), or a
bastion host.

> The SFTP (or really the SSH) RFC's are extensive and open to (wrong)
> interpretation, as abundantly demonstrated by dozens of servers and
> clients, and file transfer performance is sometimes not even half of an
> HTTPS (to avoid the dreaded F word) download or upload, which can be
> comparably secure. That also is the disadvantage of HTTPS: it can be
> made much more insecure.

(Having been in attendance at various standards meetings over the
years, and having implemented applications based on RFCs on various
occasions...) I'd be surprised if any particular set of non-trivial
RFCs actually did closely approximate reality. That's why some
implementations of the RFCs are variously considered (for better or
worse) reference implementations, and why there are "connectathon"
meetings.

> I don't think I will see something easy, secure and generally accepted
> in my lifetime.

We'll hopefully be done with the classic files-n-directories sooner or
later for all but the developers. Probably later. Ah, well.

[ChrisQ]

On 08/08/12 22:19, Stephen Hoffman wrote:

>
> ftp needs to die.
>
> But then I've mentioned my distaste for the protocol.
>

It has it's uses for the odd file transfer and it is more or
less standard across systems, which makes it quite usefull,
even decades after introduction. For serious work rsync,
wget and wput are much better...

Regards,

Chris

[gopalakrishnan]

In reply To Dirk, The difference I notice in FileZilla FTPS and FTPES is that with FTPES, Filezilla send an "AUTH TLS" as soon as the connection is established. If I enter "AUTH TLS" in OpenVMS client session (i.e. after $FTP/SSL Server port ...) I get the message "already authenticated" which means the the connection is already in explicit mode.

regards -tk

[Jose Baars]

Op donderdag 9 augustus 2012 00:41:16 UTC+2 schreef ChrisQ het volgende:

For serious work rsync,
>
> wget and wput are much better...

Unless you have to go through an NTLMvX proxy, or have they finally added that to wget etc.?

[ChrisQ]

On 08/08/12 23:08, Jose Baars wrote:

>
> Unless you have to go through an NTLMvX proxy, or have they finally
added that to wget etc.?

I've no idea and don't track development anyway.

A wget --version produces:

> GNU Wget 1.12 built on solaris2.10.
>
> +digest +ipv6 -nls +ntlm +opie +md5/solaris +https -gnutls +openssl
> -iri
>
> Wgetrc:
> /etc/wgetrc (system)
> Compile: /ws/on10-tools/SUNWspro/SOS8/bin/cc -DHAVE_CONFIG_H
> -DSYSTEM_WGETRC="/etc/wgetrc" -DLOCALEDIR="/usr/sfw/share/locale"
> -I. -I../lib -I/usr/sfw/include -xO3 -xarch=v8 -xspace -W0,-Lt
> -W2,-Rcond_elim -Xa -xildoff -xc99
> Link: /ws/on10-tools/SUNWspro/SOS8/bin/cc -xO3 -xarch=v8 -xspace -W0,-Lt
> -W2,-Rcond_elim -Xa -xildoff -xc99 /usr/sfw/lib/libssl.so
> /usr/sfw/lib/libcrypto.so -R/usr/sfw/lib -lmd5 -ldl -lsocket -lnsl
> -lrt ftp-opie.o openssl.o http-ntlm.o gen-md5.o ../lib/libgnu.a
>
> Copyright (C) 2009 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://www.gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.

Probably not the latest version though...

Regards,

Chris

[John Wallace]

Is the MS "Threat Management Gateway" one of those NTLMvx proxy
things? If so, there's at least one not entirely transparent Linux
workaround available for making the TMG work with Linux client systems
in the TMG mode where client authentication is required. No
modifications needed to browsers, wget, whatever, but you probably
have to enter your Windows authentication details (once?) for the
Linux end of the proxy. Sadly I can't remember the name of it - it's
not on one of my boxes, but I have seen it working recently.

[Paul Sture]

On Thu, 09 Aug 2012 11:30:22 +0000, ChrisQ wrote:

> I've no idea and don't track development anyway.
>
> A wget --version produces:
>
>> GNU Wget 1.12 built on solaris2.10.
>>
>> +digest +ipv6 -nls +ntlm +opie +md5/solaris +https -gnutls +openssl
>> -iri
>>
>

> Probably not the latest version though...

On an up to date Fedora 17 system:

wget --version
GNU Wget 1.13.4 built on linux-gnu.

+digest +https +ipv6 -iri +large-file +nls +ntlm +opie +ssl/openssl


--
Paul Sture

[Steven Schweda]

> > ftp needs to die.

> [...] For serious work rsync, wget and wput are much better...

Perhaps I'm confused, but I thought that FTP was a file
transfer protocol, while wget was a computer program (which
can use FTP, among other protocols).

> On an up to date Fedora 17 system:
>
> wget --version
> GNU Wget 1.13.4 built on linux-gnu.

> [...]

1.14 was recently released.

http://www.gnu.org/software/wget/wget.html
http://ftp.gnu.org/gnu/wget/

[Jose Baars]

Op donderdag 9 augustus 2012 13:45:18 UTC+2 schreef John Wallace het volgende:

If so, there's at least one not entirely transparent Linux
> workaround available for making the TMG work with Linux client systems
> in the TMG mode where client authentication is required. No
> modifications needed to browsers, wget, whatever, but you probably
> have to enter your Windows authentication details (once?) for the
> Linux end of the proxy. Sadly I can't remember the name of it - it's
> not on one of my boxes, but I have seen it working recently.

One of those workarounds would be cntlm, at
https://sourceforge.net/projects/cntlm/

[Jose Baars]

Op donderdag 9 augustus 2012 13:45:18 UTC+2 schreef John Wallace het volgende:

I have ported cntlm to VMS. Alpha PCSI kit, zip with sources and some
docs available at http://www.peut.org/vms/cntlm/index.html

[Steerpike]

I have used OpenVMS 8.3 TCPIP V5.7 ECO3 FTP/SSL to connect to (Windows Server) FILEZILLA and it works.

FILEZILLA server had to be set to EXPLICIT.

Note that initial connection is using the standard FTP port 21 and then the client requests encryption of either or both of the data and control channels.

The server controls the types of FTP connections that are permitted.

The only problem I have is that the OpenVMS FTP/SSL client session terminates after successfully completing a PUT command, and the FILEZILLA log shows that a QUIT command was sent.

[gopalakrishnan]

I finally got this going and I can transfer files (...but read on).
We had to include the following rules on the ISA:

New Protocols created:
FTPS (FTP over SSL)
Ports: 989 and 990, 5000 to 5099
Protocol Types: TCP and UDP
Direction: Outbound

Updates to Rule 36 Outbound – SFTP from OpenVMS servers to ShellCard:
FTPS (FTP over SSL) added to Protocols
‘From’ clients
Sftp.xxxx.co.nz added to ‘To’

But now....

I can pick up smaller files of a few thousand bytes. When it comes to large file, I get a VMS error "-RMS-F-RSZ, invalid record size" on the output file on a get

Log below:


$ FTP/SSL SFTP.SERVICE_PROVIDER.CO.NZ 22/USER=Z/PASSW="y"

220-Welcome to the SERVICE_PROVIDER SFTP service.

220-
220-Use of this service is subject to the security and acceptable use policies

220-of SERVICE_PROVIDER; copies of these documents can be obtained from
220 security@SERVICE_PROVIDER.co.nz for approved usage by SERVICE_PROVIDER clientelle.
Connected to SFTP.SERVICE_PROVIDER.CO.NZ.

234 Using authentication type TLS
200 PBSZ=0
200 Protection level set to P
331 Password required for z
230 Logged on

FTP> enable log
Bell off.
Debugging on (debug=1).

FTP> set passive on
Passive is ON.

FTP> cd download
---> CWD download

250 CWD successful. "/download" is current directory.

FTP> dir
---> PASV
227 Entering Passive Mode (117,120,34,121,19,171)
---> LIST
150 Connection accepted
-rw-r--r-- 1 ftp ftp 1034 Aug 15 16:29 S-0814-276.RPT.2012-08-15_16.29.31.pgp.asc.zip
-rw-r--r-- 1 ftp ftp 265667 Aug 15 16:30 SX120814.DAT.2012-08-15_16.29.57.pgp.asc.zip

226 Transfer OK
288 bytes received in 00:00:00.00 seconds (2.68 Gbytes/s)

FTP> binary
---> TYPE I
200 Type set to I

FTP> get S-0814-276.RPT.2012-08-15_16.29.31.pgp.asc.zip
---> PASV
227 Entering Passive Mode (117,120,34,121,19,172)
---> RETR S-0814-276.RPT.2012-08-15_16.29.31.pgp.asc.zip
150 Connection accepted
226 Transfer OK
local: DKA2:[PROD_USERS.THIS_USER]S-0814-276^.RPT^.2012-08-15_16^.29^.31^.pgp^.asc.zip;7 remote: S-0814-276.RPT.2012-08-15_16.29.31.pgp.asc.zip
1034 bytes received in 00:00:00.00 seconds (504.88 Kbytes/s)

FTP> binary
---> TYPE I
200 Type set to I

FTP> get SX120814.DAT.2012-08-15_16.29.57.pgp.asc.zip
---> PASV
227 Entering Passive Mode (117,120,34,121,19,180)
---> RETR SX120814.DAT.2012-08-15_16.29.57.pgp.asc.zip
150 Connection accepted
%TCPIP-E-FTP_OUTPROCF, error processing output file DKA2:[PROD_USERS.THIS_USER]SX120814^.DAT^.2012-08-15_16^.29^.57^.pgp^.asc.zip;
-RMS-F-RSZ, invalid record size
426 Connection closed; transfer aborted.

FTP> quit
---> QUIT
221 Goodbye

$

[Stephen Hoffman]

On 2012-08-22 07:13:04 +0000, gopalakrishnan said:

> I finally got this going and I can transfer files (...but read on).
> We had to include the following rules on the ISA:

You can continue to pound against incompatible software here, or you
can switch to something that works better (sftp), or you can call HP
support and log a bug and let them sort it out. Your choice.

[Dirk Munk]

Check the output file that was created for the file properties. I'm sure
it is not a binary file with 512 byte fixed size records.

gopalakrishnan wrote:
> I finally got this going and I can transfer files (...but read on).
> We had to include the following rules on the ISA:
>
> New Protocols created:
> FTPS (FTP over SSL)
> Ports: 989 and 990, 5000 to 5099
> Protocol Types: TCP and UDP
> Direction: Outbound
>

> Updates to Rule 36 Outbound – SFTP from OpenVMS servers to ShellCard:

> FTPS (FTP over SSL) added to Protocols

> ‘From’ clients
> Sftp.xxxx.co.nz added to ‘To’

[Dirk Munk]

Stephen Hoffman wrote:
> On 2012-08-22 07:13:04 +0000, gopalakrishnan said:
>
>> I finally got this going and I can transfer files (...but read on).
>> We had to include the following rules on the ISA:
>
> You can continue to pound against incompatible software here, or you can
> switch to something that works better (sftp), or you can call HP support
> and log a bug and let them sort it out. Your choice.
>

Years back I tried to use SFTP on VMS. It wasn't a success, SCP worked
much better. I'm sure that at the time SFTP transferred in ASCII mode,
now I read that it is transferring in binary mode, very odd.

[Simon Clubley]

In the absence of further information, I am assuming you are using
the TCP/IP sftp client.

Be aware that sftp has undergone incompatible changes in this area
between versions.

What it actually did in older versions was to convert the file to
stream-lf internally and send that to the remote server (assuming you
were sending files _from_ VMS).

What it does in recent versions depends on the ascii/binary mode settings
in use. Note: don't assume that just because (1) the ascii/binary mode
commands are documented in the sftp online help and (2) are accepted by
the sftp client, that they actually do anything.

The commands were documented as been available before they actually did
become available.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world

[Bob Koehler]

In article <a2f9f$5034ee5f$5ed43c14$24...@cache80.multikabel.net>, Dirk Munk <mu...@home.nl> writes:
>
> Years back I tried to use SFTP on VMS. It wasn't a success, SCP worked
> much better. I'm sure that at the time SFTP transferred in ASCII mode,
> now I read that it is transferring in binary mode, very odd.

SFTP originally did not have an ASCII mode. I just sent a bunch of
bytes from one computer to the other and you got to figure out
what to do with them after they got there.

[Dirk Munk]

Simon Clubley wrote:
> On 2012-08-22, Dirk Munk <mu...@home.nl> wrote:
>> Stephen Hoffman wrote:
>>> On 2012-08-22 07:13:04 +0000, gopalakrishnan said:
>>>
>>>> I finally got this going and I can transfer files (...but read on).
>>>> We had to include the following rules on the ISA:
>>>
>>> You can continue to pound against incompatible software here, or you can
>>> switch to something that works better (sftp), or you can call HP support
>>> and log a bug and let them sort it out. Your choice.
>>>
>>
>>
>> Years back I tried to use SFTP on VMS. It wasn't a success, SCP worked
>> much better. I'm sure that at the time SFTP transferred in ASCII mode,
>> now I read that it is transferring in binary mode, very odd.
>
> In the absence of further information, I am assuming you are using
> the TCP/IP sftp client.
>
> Be aware that sftp has undergone incompatible changes in this area
> between versions.
>
> What it actually did in older versions was to convert the file to
> stream-lf internally and send that to the remote server (assuming you
> were sending files _from_ VMS).

So indeed SFTP was uaing ASCII mode. Sending a file to a Windows system
most likely would have resulted in a <cr><lf> file on the Windows side
as with normal FTP ASCII transfers.

[Dirk Munk]

I don't agree, because that would be a binary transfer. At the time I
read an O'Reilly book on SSH, and it clearly stated that only ASCII
transfers were possible. Great for text files, not usable for binaries,
including JPG's etc. , at least not on VMS or between Unix and Windows
because of the <lf> to <lf><cr> conversion.

[Simon Clubley]

Depends on the operating system.

In Linux-land, the file gets transmitted as-is, which makes it binary mode.
I just transmitted a jpeg between two Linux boxes and the MD5 checksums
were identical on both systems.

In VMS-land, the TCP/IP Services sftp client originally took your file and
converted it internally to stream-lf format during transmission.

[Bob Koehler]

In article <64d7$50360db1$5ed43c14$27...@cache1.tilbu1.nb.home.nl>, Dirk Munk <mu...@home.nl> writes:
>
> So indeed SFTP was uaing ASCII mode. Sending a file to a Windows system
> most likely would have resulted in a <cr><lf> file on the Windows side
> as with normal FTP ASCII transfers.
>

I think the VMS SFTP was trying to look like it was doing binary
transfers from a Windows system, so text files would contain CRLF
line separators.

Other SFTP did nothing with the data. Transfers between UNIX and
Windows, for example, would result in the wrong line separators.

[Bob Koehler]

On 2012-08-23, Dirk Munk <mu...@home.nl> wrote:
>
> I don't agree, because that would be a binary transfer. At the time I
> read an O'Reilly book on SSH, and it clearly stated that only ASCII
> transfers were possible. Great for text files, not usable for binaries,
> including JPG's etc. , at least not on VMS or between Unix and Windows
> because of the <lf> to <lf><cr> conversion.

Wouldn't be the first time O'Reilly published a mistake.

I transfered tons of binary files using SFTP prior to its getting
an ASCII/binary switch. Transfers between UNIX and Windows always
showed line ending problems just like FTP would if I transfered
them in binary.

Reminded me of a fellow I worked with a few years earlier, who had
only Sun and Windows experience, who always insisted "binary always
works". He assumed that dos2unix and unix2dos was to be expected.

He couldn't for the life of him understand the use of site commands
with Multinet FTP server to get a proper transfer of a UNIX text
file into a VMS fixed length record format.

[Steerpike]

I have used OpenVMS 8.3 TCPIP V5.7 ECO3 FTP/SSL to connect to (Windows Server) FILEZILLA and it works but with limitations:

1. After successful ASCII mode PUT the session terminates

2. BINARY mode PUT works as expected

3. ASCII mode GET works as expected

4. BINARY mode get fails with:
%TCPIP-E-FTP_OUTPROCF, error processing output file xxx -RMS-F-RSZ, invalid record size

[Stephen Hoffman]

This question is certainly popular. Looks like we have a whole variety
of folks working on the same problem indepenedently, and nobody has a
support contract with HP, or nobody's willing to use the support
contract to report the problem to HP and have them sort this out.
(Well, that or FTP/SSL and Filezilla have just become the most popular
file transfer sequence on the planet...)

Shove FTP/SSL in a box somewhere, and use SFTP. It's increasingly
clear from all the postings and from all the folks that are working on
this case, that something is broken here; either with Filezilla or with
the OpenVMS TCP/IP Services. (And there have definitely been recent
bugs with TCP/IP Services. This may well be another one.)

And in general, ftp is designed to be incompatible with firewalls, and
to corrupt RMS files, and particularly when transfering OpenVMS files
to disparate systems. Either convert the file to a format that the
target platform will properly process (stream LF is usually fairly
safe), or always zip "-V" the OpenVMS file(s) before tossing the
OpenVMS files around via FTP.

[Dirk Munk]

I suppose the binary get will open a file, even if it can't write (all
of) the data.
What does anal/rms of the output show? Is it a binary file (512 byte
fixed size records) ?

[Philippe...@laposte.net]

A possible another approach when dealing with cross-companies files exchanges and all burden with firewalls protected ports is to use either an HTTP tunnel or a SOCKS server solutions.

I describe these two possible approaches at http://vouters.dyndns.org/tima/Unix-OpenVMS-httptunnel-Setting_an_HTTP_tunnel_between_two_applications.html

The HTTP tunnel solution has been successfully tested to establish an SSH connection between an HP owned OpenVMS node acting as an SSH client within highly protected HP network and my private Linux computer behind my firewalls (Linux and DSL box) acting as an SSH server.

The other SOCKS approach has finally be kept and successfully used by the HP customer I indirectly worked for via an acquaintance within HP support.

In the hope this may help you to reach your objective.
Best,
Philippe

[Steerpike]

Ana/rms reveals no errors in the fragment of binary file created and, yes, the file is fixed 512.

Given the choice of how to get from A to B probably no one would start from here (including using OpenVMS or HP) however neither SFTP nor secure HTTP are options for the current project.

FTP/SSL is new with 5.7 and so perhaps problems are not entirely unexpected these days.

Perhaps extending a very widely implemented standard (FTP) however creaky to add security (FTPS (FTP/SSL)) may be seen by many folk as preferable to introducing a whole new standard (SFTP).

[Stephen Hoffman]

On 2012-08-28 18:05:09 +0000, Steerpike said:

> Ana/rms reveals no errors in the fragment of binary file created and,
> yes, the file is fixed 512.
>
> Given the choice of how to get from A to B probably no one would start
> from here (including using OpenVMS or HP) however neither SFTP nor
> secure HTTP are options for the current project.

How quaint that you think that.

There are always options.

I can think of several, not the least of which is sneaker-net with CD
or DVD media, or swapping IP stacks, or by counseling or replacing the
people that are impeding the corporate goals here through strict
adherence to incompatible requirements.

Escalate this to your management. These red-tape messes are what
management gets paid to deal with.

> FTP/SSL is new with 5.7 and so perhaps problems are not entirely
> unexpected these days.

Contact HP support.

> Perhaps extending a very widely implemented standard (FTP) however
> creaky to add security (FTPS (FTP/SSL)) may be seen by many folk as
> preferable to introducing a whole new standard (SFTP).

Extending an ancient mess of a protocol and that's actually older than
IP itself - and that centrally features a design that's inherently
incompatible with modern network security implementations - by
strapping a bag of SSL ugliness onto the side? Uh-huh. Color me
exceedingly skeptical.

And particularly when the results of this FTP/SSL (FTPS) hackage do not
function here?

[Dirk Munk]

It would be my guess that VMS is asking for the next 512 byte and is
offered 511 or 513 byte. But why? Perhaps the record seperator <lf> is
not counted correctly?