cygwin scp -r fails
gr...@post.pl
I just installed the latest cygwin on my winXP laptop, and I am having
an issue using scp -r. I have tried it to multiple Linux and Solaris
boxes, so I am quite sure it's not on the receiving server side.
I am not sure if this is a bug or my problem; here's what I do:
scp -rv 2006* user@unix:.
Normally this would (and did on the older version) copy all of my 2006*
folders into the home directory of the "user" on the box "unix" - what
it does now, it creates empty folders.
When the command is ran with multiple v for verbosity (and I am using
folder windows for testing here) it still does not work, and here's the
debug info.
Has anyone seen it? Any ideas? I have found 2 similar postings on the
web, but no resolutions so far.
My cygwin version info:
OpenSSH_4.3p1, OpenSSL 0.9.8a 11 Oct 2005
Cygwin version I am not sure how to tell, but the cygwin dll is
1.5.19-4
Thx in advance for any help/ideas.
GregT
------ debug info -----
bash-3.00$ scp -r -vvvv windows root@server:.
Executing: program /usr/bin/ssh host server, user root, command scp -v
-r -t
OpenSSH_4.3p1, OpenSSL 0.9.8a 11 Oct 2005
debug2: ssh_connect: needpriv 0
debug1: Connecting to server [10.10.10.10] port 22.
debug1: Connection established.
debug1: identity file /home/myuser/.ssh/identity type -1
debug1: identity file /home/myuser/.ssh/id_rsa type -1
debug1: identity file /home/myuser/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version
Sun_SSH_1.0.1
debug1: match: Sun_SSH_1.0.1 pat Sun_SSH_1.0*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_4.3
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit:
diffie-hellman-group-exchange-sha1,diffie-hellman-g
up14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit:
aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfou
28,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijnda...@lysator.liu.se,aes128
tr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit:
aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfou
28,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijnda...@lysator.liu.se,aes128
tr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit:
hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@op
ssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit:
hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@op
ssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zl...@openssh.com,zlib
debug2: kex_parse_kexinit: none,zl...@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,blowfish-cbc,3des-cbc
debug2: kex_parse_kexinit: aes128-cbc,blowfish-cbc,3des-cbc
debug2: kex_parse_kexinit: hmac-sha1,hmac-md5
debug2: kex_parse_kexinit: hmac-sha1,hmac-md5
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit:
\377\377\377\377,geo,lcttab,iso_8859_1,iso_8859_15,
_CA,en_CA.ISO8859-1,en_US,en_US.ISO8859-1,en_US.ISO8859-15,en_US.ISO8859-15@eu
,es,es_MX,es_MX.ISO8859-1,fr,fr_CA,fr_CA.ISO8859-1,en
debug2: kex_parse_kexinit:
\377\377\377\377,geo,lcttab,iso_8859_1,iso_8859_15,
_CA,en_CA.ISO8859-1,en_US,en_US.ISO8859-1,en_US.ISO8859-15,en_US.ISO8859-15@eu
,es,es_MX,es_MX.ISO8859-1,fr,fr_CA,fr_CA.ISO8859-1,en
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_init: found hmac-md5
debug1: kex: server->client aes128-cbc hmac-md5 none
debug2: mac_init: found hmac-md5
debug1: kex: client->server aes128-cbc hmac-md5 none
debug2: dh_gen_key: priv key bits set: 111/256
debug2: bits set: 522/1024
debug1: sending SSH2_MSG_KEXDH_INIT
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug3: check_host_in_hostfile: filename /home/myuser/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 4
debug3: check_host_in_hostfile: filename /home/myuser/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 4
debug1: Host 'server' is known and matches the RSA host key.
debug1: Found key in /home/myuser/.ssh/known_hosts:4
debug2: bits set: 537/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/myuser/.ssh/identity (0x0)
debug2: key: /home/myuser/.ssh/id_rsa (0x0)
debug2: key: /home/myuser/.ssh/id_dsa (0x0)
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/myuser/.ssh/identity
debug3: no such identity: /home/myuser/.ssh/identity
debug1: Trying private key: /home/myuser/.ssh/id_rsa
debug3: no such identity: /home/myuser/.ssh/id_rsa
debug1: Trying private key: /home/myuser/.ssh/id_dsa
debug3: no such identity: /home/myuser/.ssh/id_dsa
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
root@server's password:
debug3: packet_send2: adding 64 (len 57 padlen 7 extra_pad 64)
debug2: we sent a password packet, wait for reply
debug1: Authentication succeeded (password).
debug2: fd 4 setting O_NONBLOCK
debug2: fd 5 setting O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Entering interactive session.
debug2: callback start
debug2: client_session2_setup: id 0
debug1: Sending command: scp -v -r -t .
debug2: channel 0: request exec confirm 0
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 16384
debug2: channel 0: rcvd adjust 32768
Entering directory: D0770 0 windows
debug2: channel 0: read<=0 rfd 4 len 0
debug2: channel 0: read failed
debug2: channel 0: close_read
debug2: channel 0: input open -> drain
debug2: channel 0: ibuf empty
debug2: channel 0: send eof
debug2: channel 0: input drain -> closed
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug2: channel 0: obuf empty
debug2: channel 0: close_write
debug2: channel 0: output drain -> closed
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
#0 client-session (t4 r0 i3/0 o3/0 fd -1/-1 cfd -1)
debug3: channel 0: close_fds r -1 w -1 e 6 c -1
debug3: fd 0 is not O_NONBLOCK
debug3: fd 1 is not O_NONBLOCK
debug1: Transferred: stdin 0, stdout 0, stderr 0 bytes in 0.7 seconds
debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.0
debug1: Exit status 0
bash-3.00$
Darren Tucker
> I just installed the latest cygwin on my winXP laptop, and I am having
> an issue using scp -r. I have tried it to multiple Linux and Solaris
> boxes, so I am quite sure it's not on the receiving server side.
> I am not sure if this is a bug or my problem; here's what I do:
> scp -rv 2006* user@unix:.
>
> Normally this would (and did on the older version) copy all of my 2006*
> folders into the home directory of the "user" on the box "unix" - what
> it does now, it creates empty folders.
> When the command is ran with multiple v for verbosity (and I am using
> folder windows for testing here) it still does not work, and here's the
> debug info.
>
> Has anyone seen it? Any ideas? I have found 2 similar postings on the
> web, but no resolutions so far.
You're running it from the Cygwin shell or a Windows command prompt?
I suspect the latter.
On Unix, the shell expands wildcards but on Windows it's left up to
individual apps.
Up until 4.3p1, scp invoked ssh via the system(), which has the side
effect of expanding wildcards (since it uses the shell). 4.3p1 and
newer don't use system(), instead they construct an arglist and invoke
execl directly.
So, given a directory containing the files "foo" and "bar", on Unix
the command "scp * host:" will be expanded to "scp foo bar host:", scp
will open "foo" and "bar" in turn and send them to "host". On the
Windows CLI, the "*" won't be expanded and scp will try to open a
file literally called "*" which doesn't exist.
A workaround is to have the Cygwin shell expand the wildcards, eg:
c:\> sh -c "scp -r 2006* server:"
--
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.
gr...@post.pl
difference. Single (or multiple) files are just fine, the problem is
only occuring when I try to do a recursive copy of a directory.
Just an empty directory gets created.
I never get any error that the file was not found.
Thx for your suggestion though; I am sure it will save me a headache
some time ;)
GregT
Claudiu Costin
Have you verified if on the remote filesystem you have enough space?
>
> GregT
--
kind regards,
Claudiu Costin
gr...@post.pl
is an issue. Permissions, I do not see how either, as I am scping as
root.
The only other suspicion would be the windows rights... but in cygwin I
can get into these directories and cat files out..... or cp locally,
including cp -r, so... I am really at a loss.
I wonder if I should look for another version of opensshm but I was
hoping not to bother with this so much.. Last time I installed about a
year ago everything just worked flawlessly, so I did not anticipate
this becoming such a project.
BTW, on the windows system I also have enough disk space - even if some
huge temp files were needed....
I think I will fail over to do ssh and tar with a pipe inside, it's
just that scp used to be so convenient.
Well; if anyone else has any ideas I am all eyes ;)
I take it the debug info did not help much... If there's a command I
can run to get more info captured I'd be happy to do so.
Thx.
GregT
andrew.s...@gmail.com
anymore...
test:
open an xterm...
$ which ssh
/usr/bin/ssh
$ mkdir test
$ echo "test" > test/test.dat
$ scp -r test remotehost:~/
$ ssh remotehost ls -l test
total: 0
The directory gets created on remotehost, but no files are copied!
Yay!! :( :( :(
pls pls let's get this fixed...
gr...@post.pl
boxes it's working fine; so certainly the problem is with cygwin or
certain people's setup.
In the process I tried to use a workaround that I would use (and it
works) on solaris. Well, it didn;t work at first, but eventually did
once I went into ksh in cygwin (seems like sh or bash, whatever the
default is does not like something there)
anyhow, this worked for me:
ksh
/cygdrive/c/Documents and Settings/admin/My Documents
tar -cf - 2006* | ssh user@server "cd public_html; tar -xvf - "
So.. just for the time being, this does work, and does the job...
I am sort of happy to see someone else has the issue, but I'd rather
see the resolution...
GregT
Darren Tucker
> I tried, and runnish it from sh or bash or even cmd prompt makes no
> difference. Single (or multiple) files are just fine, the problem is
> only occuring when I try to do a recursive copy of a directory.
> Just an empty directory gets created.
I wasn't able to reproduce it from your description. Could somone please
email me a tarball of a directory that exhibits the problem? Or better
yet, open a bug at bugzilla.mindrot.org and attach the tarball?
gr...@post.pl
Simply scp with the -r does not work any which way I've tried it.
I take it that Darren can copy recursively no problem?
I just reinstalled cygwin, no change.
I'll go on and submit a bug, will see what will happen.
gr...@post.pl
http://bugzilla.mindrot.org/show_bug.cgi?id=1161
Darren Dunham
> Hi.
> I just installed the latest cygwin on my winXP laptop, and I am having
I just upgraded mine.
$ ssh -V
OpenSSH_4.3p1, OpenSSL 0.9.8a 11 Oct 2005
> ------ debug info -----
[snip]
> Entering directory: D0770 0 windows
> debug2: channel 0: read<=0 rfd 4 len 0
> debug2: channel 0: read failed
That part seems odd..
I wonder if you can run it under strace and see what's happening when it
goes into the directory.
On my machine between the "Entering directory" and the "read<=0" I
actually copy the file... Mine is just "file" under "dir".
[snip...]
Entering directory: D0755 0 dir
debug2: channel 0: rcvd ext data 18
Sink: D0755 0 dir
debug2: channel 0: written 18 to efd 6
Sending file modes: C0644 0 file
debug2: channel 0: rcvd ext data 19
Sink: C0644 0 file
debug2: channel 0: written 19 to efd 6
file 100% 0 0.0KB/s 00:00
debug2: channel 0: rcvd ext data 8
Sink: E
debug2: channel 0: written 8 to efd 6
debug2: channel 0: read<=0 rfd 4 len 0
[snip]
--
Darren Dunham ddu...@taos.com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
Darren Tucker
> This is ANY directory.
> Simply scp with the -r does not work any which way I've tried it.
> I take it that Darren can copy recursively no problem?
Yep. 4.3p2 on both ends and scp -r works fine for me.
I think Darren Dunham is on the right track (with the "read failed" thing)
but I have no idea why it affects some people and not others. Maybe a
cygwin thing binary/ascii thing? What's the CYGWIN env var on the problem
machines?
gr...@post.pl
Good idea.
Comming from solaris i got so used to truss, that I did not even think
of strace. Not to mention, I really did not expect CygWin to be that
unixlike!
Anyhow, I have generated a trace and added it to the bug. I cannot make
enough sense out of it at the moment as I cannot compare it to a
working one, but I agree, this is the right track to take.
Thx for the idea!
GregT
Per Hedeland
gr...@post.pl writes:
>Thx!
>
>Good idea.
http://cfaj.freeshell.org/google/
--Per Hedeland
p...@hedeland.org
les.ma...@gmail.com
which were working find are now failing to copy files within
subdirectories for -r. I'd say its a nice, hard failure... Anything I
can contribute to the resolution?
les at ivsds dot com
Darren Tucker
> I'm also having this problem with the latest scp on Cygwin: scripts
> which were working find are now failing to copy files within
> subdirectories for -r. I'd say its a nice, hard failure... Anything I
> can contribute to the resolution?
I think we've figured it out: Cygwin's struct dirent changed (the d_ino
member doesn't exist any more) and I think the bundled scp is built
against the old headers.
There's a patch attached to the bug, you could try that.
http://bugzilla.mindrot.org/show_bug.cgi?id=1161
gr...@post.pl
I see tha patch at the bugzilla.
I assume that I would have to get the source of openssh and recompile
it after applying the patch?
I am not sure how this process works on CygWin.
Thx for figuring it out.
GT
Darren Tucker
> I assume that I would have to get the source of openssh and recompile
> it after applying the patch?
Yes. If you apply the second one you will also need to run "autoconf"
to rebuild configure.
> I am not sure how this process works on CygWin.
Install the prereq packages (they're listed in README.platform and
contrib/cygwin/README), apply the patch to the unpacked source and
the "./configure && make".
> Thx for figuring it out.
You're welcome. I'll mention it to the cygwin folks.