Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Invoking rsync from a script with null parameters
0 views
Skip to first unread message
Joe
Nov 4, 2012, 10:23:21 PM11/4/12
to
I'm working on a bash backup script using rsync.
(kubuntu precise 12.04, rsync 3.0.9-1ubuntu1)
To avoid having a number of slightly different rsync commands, I would
like to use shell variables as part of the rsync command. I.e.:
DRYRUN="-n"
rsync "${DRYRUN}" more parameters ...
This does not work if DRYRUN="" - apparently because this command becomes
rsync "" more parameters ...
instead of
rsync more parameters ...
and rsync uses the null parameter for something and does not perform as
expected. It does not generate any error or diagnostic message.
Is there a way to get around this problem - other than coding each
permutation of the command separately?
I'm experimenting with putting the whole rsync command in a string so I
can run it after any null parameters revert to pure white space. Once I
get the quoting to work (preserving those quotes I still need), this
method should work, but it's less than elegant.
Is this a bug in rsync? (Shouldn't it at least complain/error exit if
it gets something like this that it doesn't understand?) If it is,
what's the best way to report it?
Ideally (for me anyway), I would like it to completely ignore any null
parameters, but I don't know what problems that might cause for other
people.
TIA
Joe
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
(kubuntu precise 12.04, rsync 3.0.9-1ubuntu1)
To avoid having a number of slightly different rsync commands, I would
like to use shell variables as part of the rsync command. I.e.:
DRYRUN="-n"
rsync "${DRYRUN}" more parameters ...
This does not work if DRYRUN="" - apparently because this command becomes
rsync "" more parameters ...
instead of
rsync more parameters ...
and rsync uses the null parameter for something and does not perform as
expected. It does not generate any error or diagnostic message.
Is there a way to get around this problem - other than coding each
permutation of the command separately?
I'm experimenting with putting the whole rsync command in a string so I
can run it after any null parameters revert to pure white space. Once I
get the quoting to work (preserving those quotes I still need), this
method should work, but it's less than elegant.
Is this a bug in rsync? (Shouldn't it at least complain/error exit if
it gets something like this that it doesn't understand?) If it is,
what's the best way to report it?
Ideally (for me anyway), I would like it to completely ignore any null
parameters, but I don't know what problems that might cause for other
people.
TIA
Joe
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Kevin Korb
Nov 4, 2012, 10:35:04 PM11/4/12
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I suspect you are missing a space somewhere and are ending up with 2
parameters stuck together. You can have bash output the rsync command
line it intends to run to make sure or you can just use " " instead of
"" as extra spaces between parameters will have no effect.
- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Ke...@FutureQuest.net (work)
Orlando, Florida k...@sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
iEYEARECAAYFAlCXM+cACgkQVKC1jlbQAQdBIwCgzo0nlqGUJ4hGn/HKIsdhjHxR
vpMAoPJmq1WwiGC5gU1gQ3Wpq3xjC8lz
=rVpT
-----END PGP SIGNATURE-----
Hash: SHA1
I suspect you are missing a space somewhere and are ending up with 2
parameters stuck together. You can have bash output the rsync command
line it intends to run to make sure or you can just use " " instead of
"" as extra spaces between parameters will have no effect.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Ke...@FutureQuest.net (work)
Orlando, Florida k...@sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
iEYEARECAAYFAlCXM+cACgkQVKC1jlbQAQdBIwCgzo0nlqGUJ4hGn/HKIsdhjHxR
vpMAoPJmq1WwiGC5gU1gQ3Wpq3xjC8lz
=rVpT
-----END PGP SIGNATURE-----
Joe
Nov 5, 2012, 2:39:06 AM11/5/12
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
No. I traced the heck out of it (bash -vx ...) and I was actually
sending a null parameter as the first argument to rsync and that
made it get out of bed on the wrong side.
Here are some more details as to what happened:
http://askubuntu.com/questions/112717/rsync-copies-files-from-working-directory-in-addition-to-the-requested-ones/113431
Essentially, rsync worked, but additionally processed all the files in the current working directory when that wasn't expressly requested. Since null parameters are almost totally invisible, it took awhile to find.
Sorry I didn't include more details in my original post.
On the bright side, this is what caused me to join this list. I've learned a lot - mostly by lurking.
Thanks.
Joe
Here are some more details as to what happened:
http://askubuntu.com/questions/112717/rsync-copies-files-from-working-directory-in-addition-to-the-requested-ones/113431
Essentially, rsync worked, but additionally processed all the files in the current working directory when that wasn't expressly requested. Since null parameters are almost totally invisible, it took awhile to find.
Sorry I didn't include more details in my original post.
On the bright side, this is what caused me to join this list. I've learned a lot - mostly by lurking.
Thanks.
Joe
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Version: GnuPG v1.4.11 (GNU/Linux)
iQEcBAEBAgAGBQJQl2zyAAoJELWM3hHMxTQOhfQH+wWnHVU/b9PcvdH76fdR3E5O
/Fu12vOR5+E8oX0HTWnAF+mLX6u7GpBt1YcEQNJQ4KggFtXeMfkkALY0bkvJJ90d
tje/MAM8K7paH41QAYmUmnrWLa7iQ7kM2CM+9eHIg89sfyt71LlanvKctKY+CXuK
4Ny4DbQiqzD6cfD1CsvXjo2v87XoBgX9JS4zDSLSPWYkXm75vU1PUhT7LO1RC5zl
NETiRNXFRFEhYE2Zk3qvg2p88NcUHPg8xsMK2J2UeLDcV8pMl7DLFmps1ObjD943
2YpKVvdrc/Nfm79HxCGh36jCOJqscVNbk70/UN9raKmRqE7II1nsZNWulSrGnu4=
=4KQM
-----END PGP SIGNATURE-----
/Fu12vOR5+E8oX0HTWnAF+mLX6u7GpBt1YcEQNJQ4KggFtXeMfkkALY0bkvJJ90d
tje/MAM8K7paH41QAYmUmnrWLa7iQ7kM2CM+9eHIg89sfyt71LlanvKctKY+CXuK
4Ny4DbQiqzD6cfD1CsvXjo2v87XoBgX9JS4zDSLSPWYkXm75vU1PUhT7LO1RC5zl
NETiRNXFRFEhYE2Zk3qvg2p88NcUHPg8xsMK2J2UeLDcV8pMl7DLFmps1ObjD943
2YpKVvdrc/Nfm79HxCGh36jCOJqscVNbk70/UN9raKmRqE7II1nsZNWulSrGnu4=
=4KQM
-----END PGP SIGNATURE-----
Voelker, Bernhard
Nov 5, 2012, 3:22:56 AM11/5/12
to
Joe wrote (Monday, November 05, 2012 4:23 AM):
> To avoid having a number of slightly different rsync commands, I would
> like to use shell variables as part of the rsync command. I.e.:
> DRYRUN="-n"
> rsync "${DRYRUN}" more parameters ...
Just omit the quotes (and in this case you can
> To avoid having a number of slightly different rsync commands, I would
> like to use shell variables as part of the rsync command. I.e.:
> DRYRUN="-n"
> rsync "${DRYRUN}" more parameters ...
also omit the curly braces):
$ rsync $DRYRUN more parameters ...
Have a nice day,
Berny
Kevin Korb
Nov 5, 2012, 11:06:55 AM11/5/12
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Very interesting. It does appear to take '' or "" as if it were "./"
Hash: SHA1
in fact it appears to tale " " or ' ' as "./ " which throws an error.
I think the reason I haven't seen this before is that I always built
up a $RSYNC_PARAMS variable and passed that to rsync. That variable
always had some params in it just they weren't always the same.
On 11/05/12 02:39, Joe wrote:
>
> No. I traced the heck out of it (bash -vx ...) and I was actually
> sending a null parameter as the first argument to rsync and that
> made it get out of bed on the wrong side.
>
> Here are some more details as to what happened:
>
> http://askubuntu.com/questions/112717/rsync-copies-files-from-working-directory-in-addition-to-the-requested-ones/113431
>
> Essentially, rsync worked, but additionally processed all the
> files in the current working directory when that wasn't expressly
> requested. Since null parameters are almost totally invisible, it
> took awhile to find.
>
> Sorry I didn't include more details in my original post.
>
> On the bright side, this is what caused me to join this list.
> I've learned a lot - mostly by lurking.
>
> Thanks. Joe
>
> On 11/04/2012 10:35 PM, Kevin Korb wrote:
>> I suspect you are missing a space somewhere and are ending up
>> with 2 parameters stuck together. You can have bash output the
>> rsync command line it intends to run to make sure or you can just
>> use " " instead of "" as extra spaces between parameters will
>> have no effect.
>
>> On 11/04/12 22:23, Joe wrote:
>>> I'm working on a bash backup script using rsync. (kubuntu
>>> precise 12.04, rsync 3.0.9-1ubuntu1)
>
>>> To avoid having a number of slightly different rsync commands,
>>> I would like to use shell variables as part of the rsync
>>> command. I.e.: DRYRUN="-n" rsync "${DRYRUN}" more parameters
>>> ...
>
>>> I would like to use shell variables as part of the rsync
>>> command. I.e.: DRYRUN="-n" rsync "${DRYRUN}" more parameters
>>> ...
>
>>> This does not work if DRYRUN="" - apparently because this
>>> command becomes rsync "" more parameters ... instead of rsync
>>> more parameters ...
>
>>> and rsync uses the null parameter for something and does not
>>> perform as expected. It does not generate any error or
>>> diagnostic message.
>
>>> Is there a way to get around this problem - other than coding
>>> each permutation of the command separately?
>
>>> I'm experimenting with putting the whole rsync command in a
>>> string so I can run it after any null parameters revert to pure
>>> white space. Once I get the quoting to work (preserving those
>>> quotes I still need), this method should work, but it's less
>>> than elegant.
>
>>> Is this a bug in rsync? (Shouldn't it at least complain/error
>>> exit if it gets something like this that it doesn't
>>> understand?) If it is, what's the best way to report it?
>
>>> Ideally (for me anyway), I would like it to completely ignore
>>> any null parameters, but I don't know what problems that might
>>> cause for other people.
>
>>> TIA
>
>>> Joe
>
>
>
>>> command becomes rsync "" more parameters ... instead of rsync
>>> more parameters ...
>
>>> and rsync uses the null parameter for something and does not
>>> perform as expected. It does not generate any error or
>>> diagnostic message.
>
>>> Is there a way to get around this problem - other than coding
>>> each permutation of the command separately?
>
>>> I'm experimenting with putting the whole rsync command in a
>>> string so I can run it after any null parameters revert to pure
>>> white space. Once I get the quoting to work (preserving those
>>> quotes I still need), this method should work, but it's less
>>> than elegant.
>
>>> Is this a bug in rsync? (Shouldn't it at least complain/error
>>> exit if it gets something like this that it doesn't
>>> understand?) If it is, what's the best way to report it?
>
>>> Ideally (for me anyway), I would like it to completely ignore
>>> any null parameters, but I don't know what problems that might
>>> cause for other people.
>
>>> TIA
>
>>> Joe
>
>
>
- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Ke...@FutureQuest.net (work)
Orlando, Florida k...@sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Ke...@FutureQuest.net (work)
Orlando, Florida k...@sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Version: GnuPG v2.0.19 (GNU/Linux)
iEYEARECAAYFAlCX5B8ACgkQVKC1jlbQAQex4wCfVsvSAj12z/xZYtAUOwJtWWS1
hdcAn0SUcRTr3HXqFpQdWtSDHa/GR+q+
=kko1
-----END PGP SIGNATURE-----
Joe
Nov 5, 2012, 11:31:13 AM11/5/12
to
On 11/05/2012 03:22 AM, Voelker,
Bernhard wrote:
Joe wrote (Monday, November 05, 2012 4:23 AM):
To avoid having a number of slightly different rsync commands, I would like to use shell variables as part of the rsync command. I.e.: DRYRUN="-n" rsync "${DRYRUN}" more parameters ...
Thanks. Blindingly obvious, so I didn't see it.Just omit the quotes (and in this case you can also omit the curly braces): $ rsync $DRYRUN more parameters ... Have a nice day, Berny
I almost always put the curly braces in. They don't hurt anything and if I modify a script later, variables are protected from being accidentally renamed by something getting concatenated to them.
However, it still looks like rsync did something unexpected that it probably should not have done.
Joe
Joe
Nov 5, 2012, 11:46:19 AM11/5/12
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Thanks for duplicating this. Even though it's a very small issue,
it would be good to fix it (make it an error) because this almost
invisible scripting error can lead to very unpredictable results
just depending on the current working directory which shouldn't have
any impact at all unless it is explicitly referenced in the
parameters.
Your approach gets all the substitutions done before rsync gets to see it, so "sometimes a blank is really just a blank" ;) .
Joe
Your approach gets all the substitutions done before rsync gets to see it, so "sometimes a blank is really just a blank" ;) .
Joe
> >>> To avoid having a number of slightly different
rsync commands,
> >>> I would like to use shell variables as part of the rsync
> >>> command. I.e.: DRYRUN="-n" rsync "${DRYRUN}" more parameters
> >>> ...
>
> >>> I would like to use shell variables as part of the rsync
> >>> command. I.e.: DRYRUN="-n" rsync "${DRYRUN}" more parameters
> >>> ...
>
> >>> This does not work if DRYRUN="" - apparently
because this
> >>> command becomes rsync "" more parameters ... instead of rsync
> >>> more parameters ...
>
> >>> and rsync uses the null parameter for something and does not
> >>> perform as expected. It does not generate any error or
> >>> diagnostic message.
>
> >>> Is there a way to get around this problem - other than coding
> >>> each permutation of the command separately?
>
> >>> I'm experimenting with putting the whole rsync command in a
> >>> string so I can run it after any null parameters revert to pure
> >>> white space. Once I get the quoting to work (preserving those
> >>> quotes I still need), this method should work, but it's less
> >>> than elegant.
>
> >>> Is this a bug in rsync? (Shouldn't it at least complain/error
> >>> exit if it gets something like this that it doesn't
> >>> understand?) If it is, what's the best way to report it?
>
> >>> Ideally (for me anyway), I would like it to completely ignore
> >>> any null parameters, but I don't know what problems that might
> >>> cause for other people.
>
> >>> TIA
>
> >>> Joe
>
>
>
>
> >>> command becomes rsync "" more parameters ... instead of rsync
> >>> more parameters ...
>
> >>> and rsync uses the null parameter for something and does not
> >>> perform as expected. It does not generate any error or
> >>> diagnostic message.
>
> >>> Is there a way to get around this problem - other than coding
> >>> each permutation of the command separately?
>
> >>> I'm experimenting with putting the whole rsync command in a
> >>> string so I can run it after any null parameters revert to pure
> >>> white space. Once I get the quoting to work (preserving those
> >>> quotes I still need), this method should work, but it's less
> >>> than elegant.
>
> >>> Is this a bug in rsync? (Shouldn't it at least complain/error
> >>> exit if it gets something like this that it doesn't
> >>> understand?) If it is, what's the best way to report it?
>
> >>> Ideally (for me anyway), I would like it to completely ignore
> >>> any null parameters, but I don't know what problems that might
> >>> cause for other people.
>
> >>> TIA
>
> >>> Joe
>
>
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEcBAEBAgAGBQJQl+1MAAoJELWM3hHMxTQOGU8H/3J5jsMe1QqcSfnhiR1Y3PNK
RseQLv1CMpxX6pQom9rxmpOWaH9IhgmZxt8bPkb9qsWa7HMnaGC1o3xPq8V9fSdo
HKKXiy+V9AlEqyFIVR+rNh1Qdcc9Gs3h5nb3gMWFEga8fTTN9FzEqzEZYKrIA0MQ
qDm03iOkV3XjlGygUFAUeMUe0zJA5evqgRFhzqPRAjT2nzyH6hjBZ3MHmuk94mMQ
mIiYILD0nx3EQpFnUXJSYqa4861R8tV1u+7t9JyKMUWY00MWZ/8Go4UZyn9kjfdQ
TAz7cXqsZm1xAaNDahEM5muZ2lEdv99pAmUjpHsX6m1wHw6KqvTw8FlGKPB7M6Y=
=IPqG
-----END PGP SIGNATURE-----
RseQLv1CMpxX6pQom9rxmpOWaH9IhgmZxt8bPkb9qsWa7HMnaGC1o3xPq8V9fSdo
HKKXiy+V9AlEqyFIVR+rNh1Qdcc9Gs3h5nb3gMWFEga8fTTN9FzEqzEZYKrIA0MQ
qDm03iOkV3XjlGygUFAUeMUe0zJA5evqgRFhzqPRAjT2nzyH6hjBZ3MHmuk94mMQ
mIiYILD0nx3EQpFnUXJSYqa4861R8tV1u+7t9JyKMUWY00MWZ/8Go4UZyn9kjfdQ
TAz7cXqsZm1xAaNDahEM5muZ2lEdv99pAmUjpHsX6m1wHw6KqvTw8FlGKPB7M6Y=
=IPqG
-----END PGP SIGNATURE-----
Kevin Korb
Nov 5, 2012, 11:48:15 AM11/5/12
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Well, I am thinking that this is actually expected behavior that I had
Hash: SHA1
just never seen before. Even cat and ls duplicate this behavior.
- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Ke...@FutureQuest.net (work)
Orlando, Florida k...@sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Ke...@FutureQuest.net (work)
Orlando, Florida k...@sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Version: GnuPG v2.0.19 (GNU/Linux)
iEYEARECAAYFAlCX7c8ACgkQVKC1jlbQAQe6KACgxxFlf9E+zYAe9xfEkMdzZAKR
WEsAoLh6+08EKM2QG2BoUpUnMFch6Lcm
=KMuV
Voelker, Bernhard
Nov 6, 2012, 6:01:20 AM11/6/12
to
Kevin Korb wrote (Monday, November 05, 2012 5:48 PM):
> Well, I am thinking that this is actually expected behavior that I had
> just never seen before. Even cat and ls duplicate this behavior.
Not quite: while the coreutils programs issue an error message for
> Well, I am thinking that this is actually expected behavior that I had
> just never seen before. Even cat and ls duplicate this behavior.
zero-lenght file names as it's required by POSIX, rsync does not.
$ ls -l ''
/bin/ls: : No such file or directory
$ mkdir a b && cd a && touch f && rsync '' -ai . ../b/.
>f+++++++ f
$ rsync -ai '' ../b/.
$
Have a nice day,
Berny
0 new messages