Automatic ftp transfers

[Dave Humphreys]

We have a requirement in our application to send confirmations of Booking In and Booking out activity via email and/or ftp.

For email purposes we use ACTFAX which simply requires a file with the email information  to be written to a named folder on the server .

eg

ACTFAX.EMAIL        = ""

ACTFAX.EMAIL<-1> = @F111 email from

ACTFAX.EMAIL<-1> = @F212 email to

ACTFAX.EMAIL<-1> = @F606 attachment

ACTFAX.EMAIL<-1> = @F307 subject

WRITE ACTFAX.EMAIL  on  folder , MSG.ID

ACTFAX polls this folder extracts the email parameters and sends the email . A client screen allows you to check the status of any email.

From the application point of view once the file is written  the email process is complete.

Does anyone know of any ftp software that will work in the same way ie enter the ftp information , host / user /password etc into a file , put the file into a named folder and the software will poll and extract and send send the ftp file ,ideally with  a client to check the status of ftp messages.

Application runs on Windows server.

Thanks

[Kevin Powick]

I don't know of FTP software that will work in this way, but you can easily use the command line FTP client in Windows to do the job directly from your program.

1) Write your data file to a Windows folder

2) Write a text file containing the list of FTP commands required (open, cd, put, etc) to the same, or another, folder.

3) Invoke the FTP client using the "s" option to include your command script created in step 2 (e.g. !FTP -is:MyCmds.txt)

http://www.nsftools.com/tips/MSFTP.htm

Kevin Powick

[George Gallen]

I ran into problems trying to trigger the ftp by command on windows

It would fire off the "is it ok for .... to access the internet" on the server machine, and wound up locking up the UV server until it was rebooted.

I forget what the issue was, it was discussed I believe on this list or one of variant UV lists but it had to do with permissions and the CMD process

   when it's spawned by the UV process.

George

---

From: mvd...@googlegroups.com <mvd...@googlegroups.com> on behalf of Kevin Powick <kpo...@gmail.com>
Sent: Friday, September 9, 2016 1:11 PM
To: Pick and MultiValue Databases
Subject: [mvdbms] Re: Automatic ftp transfers

 

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com
To unsubscribe, email to: mvdbms+un...@googlegroups.com
For more options, visit http://groups.google.com/group/mvdbms

[Kevin Powick]

Sounds like a UAC (permissions) issue.

I have a client doing FTP transfers as I described using D3 9.x on Windows server 2012 R2.

--

Kevin Powick

[Joe Sheldon]

we use autointern that triggers accuterm atcs file. it runs a program that triggers a pick program that dumps data files to a local ftp using ftd. 

clients can then download yet files

Get Outlook for iOS

_____________________________
From: Dave Humphreys <da...@bealdavis.co.uk>
Sent: Friday, September 9, 2016 4:33 AM
Subject: [mvdbms] Automatic ftp transfers
To: Pick and MultiValue Databases <mvd...@googlegroups.com>

--

[Ross Ferris]

Our Visage data replication service can do this - can also compress or uncompress to/from zip files as part of the process, adding password protection as a trivial protection mechanism.

Cost is $500 - can roll your own, but by the time you factor in all of the other "stuff" (configure as a service, logging, stats etc) you would are streets in front

Drop me an email if you want more details & confirm operation meets ALL of your needs.

[Ross Ferris]

Accuterm FTD is NOT FTP ... and I suspect recipients in this scenario could be "anywhere" ... you also need a mechanism to guarantee save delivery to recipient - I'm not aware of a command line utility that does this, and you may have issues just detecting error conditions for a retry from command line

[Simon Verona]

Can't think of an off the shelf product that will do this. It would be reasonably simple to write - I don't know ow what version of mv you use.

For the ftp you can use the inbuilt ftp command with a script. I prefer using winscp https://winscp.net/eng/index.php

If writing a utility it could be done using a Windows scheduled task written using a Windows development language (eg dotnet).

It can also be done reasonably well using a phantom, which polls the directory and runs the ftp process...

On qm (my mv choice) you can do a mixture - run a qm program from a Windows scheduled task every few minutes to do the ftp.

Hope this helps
Simon

[Dave Humphreys]

Thanks for the responses.

We  pretty much  do what Kevin suggested. Running on Windows\UV we use the dos \c command line execution and use winscp as the command line client .

However we have had issues in the past when the ftp has not completed correctly and that's why we want to offload the execution of the   ftp process but still allow the processing to be initiated directly from the application .

Ross - I'll send you more information.

[Kevin Powick]

On Sunday, 11 September 2016 16:06:49 UTC-4, Dave Humphreys wrote:

 

However we have had issues in the past when the ftp has not completed correctly and that's why we want to offload the execution of the   ftp process but still allow the processing to be initiated directly from the application .

A quick google search found the following top result.

http://www.deskshare.com/ftp-client.aspx

This can be installed as a service that monitors a folder for changes.  I've never used it, but you can download a free trial.

--

Kevin Powick

[Dave Humphreys]

Kevin 

I've already had a look at the product and it can  monitor a folder for changes and then automatically ftp . However I think it relies on setting up the ftp commands through its client screen and linking each client to a specific folder which would mean that it could not be completely automated from the application. ie the process would be  1. In the ftp manager setup ftp details host/user/pw etc and the folder to monitor 2. from the application write files to the folder.

Regards

Dave

[Steve Trimble]

Here is a D3 subroutine that I use once the flat file(s) is written to a linux directory using 'curl'. 'curl', of course works on Windows as well as Linux.

SUBROUTINE FTP.FILES.SUB(ERRMSG,FTP.VARS,FILE.NAMES)

!

* 05-23-16 smt ftp using curl

!

*** FTP.VARS example

* FTP.VARS<01> = PATH '/home/pick/temp'

* FTP.VARS<02> = SITE 'ftp.xxxx.com'

* FTP.VARS<03> = USER 'usrname'

* FTP.VARS<04> = PSWD 'usrpswd'

* FTP.VARS<05> = TODIR  '/TOBW/' or '' for default directory on system being sent to

!

ERRMSG = ''

PATH  = FTP.VARS<01>

SITE  = FTP.VARS<02>

USER  = FTP.VARS<03>

PSWD  = FTP.VARS<04>

TODIR = FTP.VARS<05>

CRT OCONV(TIME(),"MTS"):\  sending file(s) to \:SITE

!

*** using curl to ftp (send to server)

!

* curl -T /var/powersoft/pcwtxq/dailies/DBL_20160502_14.SC ftp://ftp.xxxx.com/DBL_20160502_14.SC --user usrname:userpswd

* curl -T /var/powersoft/pcwtxq/dailies/DBL_20160502_14.SM ftp://ftp.xxxx.com/TOBW/DBL_20160502_14.SM --user usrnamet:usrpswd

!

NUM.FILES = DCOUNT(FILE.NAMES,@AM)

CRT

CRT \sending \:NUM.FILES:\ files to \:SITE

FOR II = 1 TO NUM.FILES

   FILENAME.HOST = FILE.NAMES<II,1> ; FILENAME.REM = FILE.NAMES<II,2>

   CMD = \curl --connect-timeout 15 -T \:PATH:FILENAME.HOST:\ ftp://\:SITE:TODIR:FILENAME.REM:\ --user \:USER:\:'\:PSWD:\'\

   CRT CMD

   EXECUTE \! \:CMD CAPTURING RESULTS RETURNING ERRINFO

   IF ERRINFO <> 0 THEN

      ERRMSG = \curl returned error code \:ERRINFO:\ while executing [\:CMD:\]\

      IF ERRMSG <> '' THEN ERRMSG = ERRMSG:@AM

   END

NEXT II

*

CRT OCONV(TIME(),"MTS"):\  end FTP...\

*

RETURN

END

Steve Trimble

Computerized Data Mgmt Inc

(501) 615-8674 office

(501) 772-3450 cell / text

--

You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

[fwinans]

You could probably use the scp command that comes with cygwin installs instead of using winscp
{have not tried it scp myself, thoug}.
I sometimes use pscp.exe to shovel a file or a folder tree to our linux box from my winbox;
   this is available on the putty download page...
     If you go the  pscp.exe route though,
         a)  suggest you do a manual putty.exe login to the destination server, and set that session up
             to use shared key instead of password login.  Make this use a guest account of some sort,
             as the associated private key on your winbox is a deadly security risk to that destination server!
             As in, even changing all the passwords there after you fire a disgruntled employee will still leave
            this private key working just fine!   While in the putty pre-connect screens tell it to save your
            settings as bobfoo for example.  Now your windows putty and pscp commands can be
            enhanced with the   -load  bobfoo    command line args.  Something like 
                   echo   n   |   pscp.exe   -load   bobfoo   c:\myfile.txt   bobguy@otherbox/tmp

          b)  the first time you connect to that otherbox using putty or pscp it will stop and alert you that it
               does not yet have that box's 'fingerprint' on file, and ask you for permission to proceed.
               That is part of why I echo a  "n"  into the pscp.exe  standard in in my example; it makes the
                command fail and gets you back out of the subshell instead of waiting for your keystrokes.

                Using a symbolic {ie, dns } name counts as a 'new' fingerprint if you've only ever used an
                explicit ip address in the past, so this is not as unlikely a situation as one might imagine.

                Likewise, after one does fire a disgruntled employee and change the keys on the otherbox
                ssh  side to lock out old private keys, you'll get a strident warning message on your windows
               screen about possible 'man in the middle' attacks when you pscp  or scp  or putty  to the
               otherbox, so remember to clear out the old fingerprint
               line items on this winbox to prevent/recover from that situation.  Oh, and on linux the command to
               change the ssh keys  is     ssh-keygen

           c) if you fatfinger something, like I often forget to put the      bobguy@       part on the front of the 'to'  pathname,
               it will be sneaky and claim your key was rejected and then ask you for a password,
               instead of just saying your forgot to specify what user account to use.  But if I've
              piped a  "n"  into the  pscp.exe    command standard input then it will instead complain and then fail immediately,
              which is ideal  when you're doing this shelled out  from d3.

    Erm, putty keeps config things in your windows registry, not in disk files;
   so to  review/prune fingerprint entries  plz fire up windows command     regedit
         and dig down to
HKEY_CURRENT_USER\Software\SimonTatham\PuTTY\SshHostKeys
           Sorry, just searching for the string  putty  is a bad idea since you'll get lotsa false matches on center part of
               InputType       names in other software packages...   Could search for Simon   instead, I guess...
     Actually, I guess you could access the registry using the windows    reg     command from a cmd.exe prompt,
    so your basic program could handle this housekeeping on its own, somewhat.

Sorry you've had problems with windows  UAC sticking its beak in when you launch your commands;
On win7 I'm not getting any uac prompts when I run  pscp.exe  from a cmd.exe prompt, so hopefully you can
do a cmd.exe  /c  ' echo "n" | pscp.exe  -load  bobfoo  c:/foo.txt bobguy@otherbox/tmp/foo.txt'
without making it wait for a mouse click to continue...

[Tony Gravagno]

I think Steve's solution is the best here so far.

Dave (who is already a client) - If Steve's solution doesn't suit your needs, I have a utility which does exactly what you describe from any MV app. I would have to charge a few bucks for it but it's stable and I will provide some docs and support if required.

Regards,
T

[fwinans]

This is so much easier in linux;  I guess historically in windows they expect you to have the remote computer disk share mounted as a drive letter and just use copy/del/dir commands to get things done. But lately setting up a windows print and file sharing link between two winboxes manually is fraught with surprises as they tighten down the windows security more every few years and add new features like virtual directories that outwit old established apps in your shop. They _are_ making things a little more flexible in the command-line environment than it was back in win95 days,  with nice new 'verbs' as we'd call them in tcl  {such as the REG command to access windows registry} and massive enhancements to the various  subcommands of  NET.  If we were a windows-only shop I'd personally be tempted to set up a linux server just to act as a relay point between apps on the winboxes, so that the apps wouldn't get messed with by future windows changes so badly.  Linux, like d3, changes much more slowly than windows does over the years.

[Drew Conboy]

Dave,
we have developed a custom FTP utility that does exactly what you are looking for.....
We have it running at over 100 sites.. it's very reliable
If you are interested let me know
Drew Conboy

[Wols Lists]

On 13/09/16 14:20, fwinans wrote:
> This is so much easier in linux; I guess historically in windows they
> expect you to have the remote computer disk share mounted as a drive
> letter and just use copy/del/dir commands to get things done. But lately
> setting up a windows print and file sharing link between two winboxes
> manually is fraught with surprises as they tighten down the windows
> security more every few years and add new features like virtual
> directories that outwit old established apps in your shop.

Those virtual directories are a pain in the arse !!!

Plus, they're also clamping down on networking so that, unless you pay
extra for the various enterprise versions, different versions of Windows
won't talk to each other. For example, I think I managed to get XP to
talk to 7, but not the other way.

My biggest bugbear with virtual directories is that they won't "virtual"
any directory that doesn't support Windows indexing. The "workaround" is
to mirror the - presumably network - drive onto your local drive !?!?!?
How am I supposed to mirror my 3TB network storage onto my PC's 50GB
local drive?

(The alternative is to force my - learning impaired due to illness -
wife to relearn where all her files are ... :-(

Cheers,
Wol

[Tony Gravagno]

hogwash

[Dave Humphreys]

Thanks for all the replies

Drew and  Ross if you let me have your email addresses we can talk further.

dave**pleaseremovethis**@bealdavis.co.uk

Regards

[Fred Waltman]

Late to the game. I have a couple of clients who need to ftp (and sftp) files to multiple sites on a nightly or weekly basis.  They are both windows platforms.

After generating the flat files in the required format (and writing them to a Windows directory) , I just shell out to a WinSCP script that uploads all the files in a specific directory to the appropriate FTP sever. If the FTP successfully complete the files are moved to an "uploaded" folder.

It's worked well for about 10 years or so. Not perfect but works well enough.

Fred

[Bill Crowell]

You should look at using cURL - Command Line URL. It does FTP. I use it for all sorts of automated transfers.

Bill Crowell

Pavuk Systems

[Rich Taylor, InterSystems]

Dave,

Sorry I am a little late to the party.  Cache has native capability to do both email and FTP interfaces without the need to write to external directories or programs.  HOWEVER, that probably does not help you as you are likely not based on Cache or considering a migration at this time.  What InterSystems can bring to the table is a interoperability platform called Ensemble (http://www.intersystems.com/our-products/ensemble/ensemble-overview/).  This is still overkill for the problem you posted about, but if you have broader needs for an interoperability solution or service bus architecture it is will worth a look.

Good luck with the FTP issues.

Regards,

Rich Taylor

On Friday, September 9, 2016 at 10:33:03 AM UTC-4, Dave Humphreys wrote:

[Dave Humphreys]

Just to update everyone - this is what I have done.( still in test mode though)

1.Create 2 folders (uv type19 files) in the application folder -  1 process folder / 1 data folder.

2 Create a windows task Schedule to execute a batch file periodically eg every 15 minutes. Initially this is a 'shell script' written in the process folder.

3 Within the application if a Client EDI master record is created(host/user/pw etc) a winscp script is written ,(using put  'xxx'*.txt - delete where xxx is a client prefix and -delete removes the file from the data folder once processed, ) and the batch file is updated to call this script.

4 When the application triggers an EDI message it is written to the data folder with the 'xxx' client prefix.

5 The scheduled batch file executes and process the client EDI messages in the folder by using the client prefix and then deletes them..

This should ensure that additional EDI clients can be added within the application and all client messages can be written to the same folder.

Thanks to everyone who responded.

Dave