Python, SFTP, Paramiko Question

[Mark Phillip]

Morning folks,

I have a python script that has been working for quite some time that connects to an SFTP server, grabs specific files, downloads them, and finally deletes them off of the remote server.

Everything works just fine, but we've had to point to a new SFTP server thats running Windows.

Now, for some reason, I'm able to push files, delete files, and list files in a directory, but not actually download the files.  I've tried every combination of '', '.', './', '/', '/home/', '/foldername/', I can think of with zero success.

Does this problem ring a bell for anyone?

Thanks,
Mark
http://markphillip.com

[Josh Kleinpeter]

First thing first, have you tried it manually? Is there a difference in the commands you type?

It is Windows, maybe '\' instead of '/'?

> --
> Our Web site: http://www.RefreshAustin.org/
>
> You received this message because you are subscribed to the Google Groups "Refresh Austin" group.
>
> [ Posting ]
> To post to this group, send email to Refresh...@googlegroups.com
> Job-related postings should follow http://tr.im/refreshaustinjobspolicy
> We do not accept job posts from recruiters.
>
> [ Unsubscribe ]
> To unsubscribe from this group, send email to Refresh-Austi...@googlegroups.com
>
> [ More Info ]
> For more options, visit this group at http://groups.google.com/group/Refresh-Austin

[Larry Kubin]

Haven't used the SFTP libraries, but we run our game server code
locally on windows during development. It uses Python's Twisted
Framework and we use backslashes in directory paths. In a string the
backslash "\" is used to escape characters, so you need to escape the
backslash itself with another backslash or use a raw string like so:

r'C:\My\Windows\Path'

- Larry

--
Larry Kubin

[Mark Phillip]

Thanks for the thoughts, guys.  Sadly, no luck there.  I've tried forward and backwards slashes, changing the working directory and not, and even creating new users and starting from scratch.

Because deletes and puts work just fine, I'm afraid it's some underlying Linux/Windows communication problem that I just won't be able to solve.


Thanks,
Mark
http://markphillip.com

[Kaffeen]

The first thing I would try is to make sure you are able to transfer manually. In addition, permissions and ownerships need to be examined (for files, directories, and script). Make sure umask is set and also set the sticky bit on your script (to prevent ownersip/permissions problems). As always, make sure antivirus, router, intranet, and/or firewalls are not causing problems. As a simple test - you may want to write a little script that simply tries to execute the transfer. If the existing script is written well (to handle errors, valid coding et al)...chances are it is one of the above issues (or combination). Have you tried manual FTP or testing with another computer? Does the file you are attempting to transfer exist? Are you performing a transfer in binary mode? Try zipping the file you want before transfer.

I do alot of communications programming - it is probably something deceptively simple as one of the above mentioned things. 

To rule out python issues I would certainly try some simple shell scripting to test FTP functionality.

Regards,

Jim Sparkman

Sent from my iPhone

[Ike Ofili]

Try posting on http://stackoverflow.com/ ... I bet they'd help you figure it out pretty quickly

You're going to want to use http://pastie.org/ to post your code though, so folks can see exactly what it is you might be doing wrong.

[Ian Ward]

Mark, what are the symptoms of the failure? Are you reproducing it
manually or just with your automation? Are you using the correct
modes (active/passive, bin/ascii)? Are you sure that you're using the
right kind of protocol and layer? I had hell finding a python library
that supported FTP over implicit SSL, which is different than explicit
SSL, SFTP (SSH FTP or SSH2 FTP) or TLS.

- ian

On Tue, Dec 21, 2010 at 2:09 PM, Mark Phillip <markp...@gmail.com> wrote:

[Mark Phillip]

Hey folks, thanks for the extra thoughts.

The key frustration is that this script has worked fine when communicating with our original Linux-based SFTP server--the problem has cropped up after moving to a different server.  Manually running SFTP via the command-line from the box, and via a GUI on my Windows desktop both work fine.  It's only when I run the existing Python script using Paramiko that there is a problem.

I'll try stackoverflow and then get to ripping out my last few hair follicles.


Thanks,
Mark
http://markphillip.com

[Kaffeen]

Have you run debug/verbose mode on the script and FTP? You should be able to capture the error. It sounds like your local environment ( where script lives) is at issue. Is the python script written in a universal way (I.e. No local configuration or unique variables dependent on local environment)?

Jim

Sent from my iPhone

[Ike Ofili]

Alternatively ... post your code here so folks can see exactly what you're doing.

[Mark Phillip]

Greetings folks,

My problem hasn't been solved, but I figured out that it's definitely not a Paramiko problem--most likely a problem with our vendor that set up the SFTP.

My personal lesson learned is that EC2 is awesome, and I need to use it more often.  It took me all of an hour of work to spin up a new Windows instance, install the software, and start testing with a brand new server.  It would have been half that amount of time if I didn't trip over the firewall settings.

Since I had no trouble connecting and downloading files, we know that it's not a nebulous Paramiko problem, just a simple one with the SFTP configuration.

Thanks everyone for the help.

Thanks,
Mark
http://markphillip.com