Automate rsync w/ authentication
Bryan
have set up a private/public key between the two servers so I don't
have to enter a password when using rsync. Running rsync manually
with the following command works fine:
rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
ro...@10.0.45.67:/home/bry/jquery.lookup /home/bry/tmp
But when I try to do it with python, the subprocess simply returns the
ssh -h output on stderr like I am passing some invalid syntax. What
is wrong in my translation of rsync's -e command from shell to
pythyon?
#! /usr/bin/python
from subprocess import Popen, PIPE
rsyncExec = '/usr/bin/ssh'
source = 'ro...@10.0.45.67:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
try:
p = Popen(args, stdout=PIPE, stderr=PIPE)
print 'rsync running with pid %s' % p.pid
out, err = p.communicate()
print 'Errors: %s' % err
print 'Output: %s' % out
except Exception:
print 'Error running rsync'
Chris Rebert
> I am trying to automate rsync to backup server A from server B. I
> have set up a private/public key between the two servers so I don't
> have to enter a password when using rsync. Running rsync manually
> with the following command works fine:
> rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
> ro...@10.0.45.67:/home/bry/jquery.lookup /home/bry/tmp
>
> But when I try to do it with python, the subprocess simply returns the
> ssh -h output on stderr like I am passing some invalid syntax. What
> is wrong in my translation of rsync's -e command from shell to
> pythyon?
>
> #! /usr/bin/python
> from subprocess import Popen, PIPE
> rsyncExec = '/usr/bin/ssh'
> source = 'ro...@10.0.45.67:/home/bry/jquery.lookup'
> dest = '/home/bry/tmp'
> rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
> args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
Like many problems involving the subprocess module, I think you've
tokenized the arguments incorrectly. Try:
rshArg = '"/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-av', '--dry-run', '-e', rshArg, source, dest]
Note that the -e switch and its operand are separate arguments for the
purposes of POSIX shell tokenization.
Cheers,
Chris
--
http://blog.rebertia.com
Piet van Oostrum
>CR> On Fri, Jul 10, 2009 at 9:13 AM, Bryan<brya...@gmail.com> wrote:
>>> I am trying to automate rsync to backup server A from server B. �I
>>> have set up a private/public key between the two servers so I don't
>>> have to enter a password when using rsync. �Running rsync manually
>>> with the following command works fine:
>>> rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
>>> ro...@10.0.45.67:/home/bry/jquery.lookup /home/bry/tmp
>>>
>>> But when I try to do it with python, the subprocess simply returns the
>>> ssh -h output on stderr like I am passing some invalid syntax. �What
>>> is wrong in my translation of rsync's -e command from shell to
>>> pythyon?
>>>
>>> #! /usr/bin/python
>>> from subprocess import Popen, PIPE
>>> rsyncExec = '/usr/bin/ssh'
>>> source = 'ro...@10.0.45.67:/home/bry/jquery.lookup'
>>> dest = '/home/bry/tmp'
>>> rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
>>> args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
>CR> Like many problems involving the subprocess module, I think you've
>CR> tokenized the arguments incorrectly. Try:
>CR> rshArg = '"/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
>CR> args = [rsyncExec, '-av', '--dry-run', '-e', rshArg, source, dest]
>CR> Note that the -e switch and its operand are separate arguments for the
>CR> purposes of POSIX shell tokenization.
I think you should have only one kind of quotes in rshArg:
rshArg = "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
I haven't tried it, however, but this is just how Unix works.
--
Piet van Oostrum <pi...@cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: pi...@vanoostrum.org
Chris Rebert
Ah, indeed, I think you're probably right. Just goes to show it's not
always easy to get exactly right.
Bryan
> >>>>> Chris Rebert <c...@rebertia.com> (CR) wrote:
> >CR> On Fri, Jul 10, 2009 at 9:13 AM, Bryan<bryanv...@gmail.com> wrote:
> >>> I am trying to automate rsync to backup server A from server B. I
> >>> have set up a private/public key between the two servers so I don't
> >>> have to enter a password when using rsync. Running rsync manually
> >>> with the following command works fine:
> >>> rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
>
> >>> But when I try to do it with python, the subprocess simply returns the
> >>> ssh -h output on stderr like I am passing some invalid syntax. What
> >>> is wrong in my translation of rsync's -e command from shell to
> >>> pythyon?
>
> >>> #! /usr/bin/python
> >>> from subprocess import Popen, PIPE
> >>> rsyncExec = '/usr/bin/ssh'
> >>> dest = '/home/bry/tmp'
> >>> rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
> >>> args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
> >CR> Like many problems involving the subprocess module, I think you've
> >CR> tokenized the arguments incorrectly. Try:
> >CR> rshArg = '"/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
> >CR> args = [rsyncExec, '-av', '--dry-run', '-e', rshArg, source, dest]
> >CR> Note that the -e switch and its operand are separate arguments for the
> >CR> purposes of POSIX shell tokenization.
>
> I think you should have only one kind of quotes in rshArg:
> rshArg = "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
>
> I haven't tried it, however, but this is just how Unix works.
> --
I tried removing the internal quotes, and splitting '-e' from the
other arguments. I still get the same ssh -h output however:
rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source,
dest]
Piet van Oostrum
>B> I tried removing the internal quotes, and splitting '-e' from the
>B> other arguments. I still get the same ssh -h output however:
>B> rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
>B> args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source,
>B> dest]
>B> p = Popen(args, stdout=PIPE, stderr=PIPE)
For me it works. Of course I substituted local server names, file names
etc. I run this on Mac OS X 10.4.11. The server is some Linux system.
------------------------------------------------------------------------
#! /usr/bin/env python
from subprocess import Popen, PIPE
rsyncExec = '/usr/local/bin/rsync'
source = 'xxx.cs.uu.nl:/users/piet/yyyyyy'
dest = '/Users/piet/TEMP/test.rsync'
rshArg = '/usr/bin/ssh -i /Users/piet/.ssh/id_rsa'
args = [rsyncExec, '-a', '-v', '-e', rshArg, source, dest]
try:
p = Popen(args, stdout=PIPE, stderr=PIPE)
print 'rsync running with pid %s' % p.pid
out, err = p.communicate()
print 'Errors: %s' % err
print 'Output: %s' % out
except Exception:
print 'Error running rsync'
------------------------------------------------------------------------
It just copies the file.
--
Piet van Oostrum <pi...@cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: pi...@vanoostrum.org
Gary Duzan
Bryan <brya...@gmail.com> wrote:
>
>rsyncExec = '/usr/bin/ssh'
>source = 'ro...@10.0.45.67:/home/bry/jquery.lookup'
>dest = '/home/bry/tmp'
>rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
>args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
I think you want -e and the ssh command to be separate args.
Something like:
rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest]
or:
rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ]
args = [rsyncExec, '-a', '-v', '--dry-run'] + rshArgs + [ source, dest]
Gary Duzan
Motorola H&NM
Bryan
> In article <3af970b1-b454-4d56-a33f-889ecfaca...@l28g2000vba.googlegroups.com>,> >source = 'r...@10.0.45.67:/home/bry/jquery.lookup'
> >dest = '/home/bry/tmp'
> >rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
> >args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
>
> I think you want -e and the ssh command to be separate args.
> Something like:
>
> rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
> args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest]
>
> or:
>
> rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ]
> args = [rsyncExec, '-a', '-v', '--dry-run'] + rshArgs + [ source, dest]
>
> Gary Duzan
> Motorola H&NM
Separating the argument parts worked. Strangely though, I don't need
to do that for arguments such as --files-from='/path/to/file'. Also,
in this example code I had the rsync executable path pointing to the
ssh program, so no wonder I was getting the output of ssh!
Piet van Oostrum
>B> On Jul 10, 12:03�pm, mgi...@motorola.com (Gary Duzan) wrote:
>>> In article <3af970b1-b454-4d56-a33f-889ecfaca...@l28g2000vba.googlegroups.com>,
>>>
>>> Bryan �<bryanv...@gmail.com> wrote:
>>>
>>> >rsyncExec = '/usr/bin/ssh'
>>> >source = 'r...@10.0.45.67:/home/bry/jquery.lookup'
>>> >dest = '/home/bry/tmp'
>>> >rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
>>> >args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
>>>
>>> � �I think you want -e and the ssh command to be separate args.
>>> Something like:
>>>
>>> rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
>>> args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest]
>>>
>>> or:
>>>
>>> rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ]
>>> args = [rsyncExec, '-a', '-v', '--dry-run'] + �rshArgs + [ source, dest]
>>>
>>> � � � � � � � � � � � � � � � � � � � � Gary Duzan
>>> � � � � � � � � � � � � � � � � � � � � Motorola H&NM
>B> Separating the argument parts worked. Strangely though, I don't need
>B> to do that for arguments such as --files-from='/path/to/file'. Also,
>B> in this example code I had the rsync executable path pointing to the
>B> ssh program, so no wonder I was getting the output of ssh!
I should have seen that because I changed it in my own copy!!!
--files-from='/path/to/file *is* one argument, in contrast to
-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key" which is two
arguments.