Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

passing args without eval

8 views
Skip to first unread message

Ed Morton

unread,
Jan 22, 2012, 1:21:58 PM1/22/12
to
Given this test script:

$ cat tst.sh
function sub {
printf '\tsub %s\n' "$@"
for subarg
do
printf '\t subarg = "%s"\n' "$subarg"
done
}

function main {
printf 'main %s\n' "$@"
for mainarg
do
printf ' mainarg = "%s"\n' "$mainarg"
passargs="$passargs \"$mainarg\""
done

eval sub "$passargs"
}

main "a*" "b"

$ ./tst.sh
main a*
main b
mainarg = "a*"
mainarg = "b"
sub a*
sub b
subarg = "a*"
subarg = "b"

Is there a better (simpler and/or more secure and/or whatever you think "better"
means) way to pass main()s arguments as-is down to sub() than using eval?

main() needs to parse it's arguments and pass them down in a single invocation
of sub().

In the real application main() will only pass some of it's arguments on to sub()
and some of the arguments to be passed on could be repeated multiple times, e.g.
main -f "abc" -f "def" -f "xyz" must result in a call to
sub -f "abc" -f "def" -f "xyz".

Today the loop in the real "main()" that parses arguments is implemented with
getopts.

Regards,

Ed.

Kaz Kylheku

unread,
Jan 22, 2012, 1:58:23 PM1/22/12
to
On 2012-01-22, Ed Morton <morto...@gmail.com> wrote:
> In the real application main() will only pass some of it's arguments on to sub()
> and some of the arguments to be passed on could be repeated multiple times, e.g.
> main -f "abc" -f "def" -f "xyz" must result in a call to
> sub -f "abc" -f "def" -f "xyz".

This example doesn't illustrate that the arguments passed down may be
altered, e.g.

sub -f "abc" -f "xyz" # -f "def" is gone for some reason

You may be able to use arguments in the parameter list, and then use

sub "$@"

to pass them down.

You can build positional parameter lists like this:

set -- # clear

Add an item using the Janis Papanagnou trick:

set -- "$@" "new" # extend the positional parameters with "new"

The problem is, what if you're processing the existing argument list at the
same time? You can't blow it off during processing.

For that, I have a trick of my own. Suppose that we add a sentinel word to the
existing parameters:

set -- "$@" "#EOP#" # end of params

With this marker in place, we can use shift and process stuff from the
beginning while adding stuff at the end. When we encounter "#EOP#", we
shift one more time to remove it and we're done. What's left in $1 .. $n
are the arguments to be passed down.

Stephane Chazelas

unread,
Jan 22, 2012, 2:42:42 PM1/22/12
to
2012-01-22 12:21:58 -0600, Ed Morton:
> Given this test script:
>
> $ cat tst.sh

ITYM cat tst.ksh, as you're using ksh-style definition here.
[...]

I usually do it as:

main() {
for i do
if ...; then
set -- "$@" "$i"
fi
shift
done
sub "$@"
}

BTW, it's the wrong way to use eval here. With eval, you'd want
to do it:

main() {
c=1
passargs=
for i do
if ...; then
passargs="$passargs \"\${$c}\""
fi
c=$(($c + 1))
done
eval "sub $passargs"
}

(which is going to ask something like: cmd "${1}" "${4}" "${6}"
to be evaluated which is harmless while cmd "some random strings
which might contain $, ", \, `" is less so.
--
Stephane

Ed Morton

unread,
Jan 22, 2012, 3:10:46 PM1/22/12
to
On 1/22/2012 12:58 PM, Kaz Kylheku wrote:
> On 2012-01-22, Ed Morton<morto...@gmail.com> wrote:
>> In the real application main() will only pass some of it's arguments on to sub()
>> and some of the arguments to be passed on could be repeated multiple times, e.g.
>> main -f "abc" -f "def" -f "xyz" must result in a call to
>> sub -f "abc" -f "def" -f "xyz".
>
> This example doesn't illustrate that the arguments passed down may be
> altered, e.g.
>
> sub -f "abc" -f "xyz" # -f "def" is gone for some reason

No, it doesn't, I thought it was enough to just state that that was the case.

> You may be able to use arguments in the parameter list, and then use
>
> sub "$@"
>
> to pass them down.
>
> You can build positional parameter lists like this:
>
> set -- # clear
>
> Add an item using the Janis Papanagnou trick:
>
> set -- "$@" "new" # extend the positional parameters with "new"

As soon as I do a "set --" I can no longer access the parameters to parse so a
loop like this:

set --
for arg
do
set -- "$@" "$arg"
done

wouldn't work so I don't think I can use that approach.

>
> The problem is, what if you're processing the existing argument list at the
> same time? You can't blow it off during processing.
>
> For that, I have a trick of my own. Suppose that we add a sentinel word to the
> existing parameters:
>
> set -- "$@" "#EOP#" # end of params
>
> With this marker in place, we can use shift and process stuff from the
> beginning while adding stuff at the end. When we encounter "#EOP#", we
> shift one more time to remove it and we're done. What's left in $1 .. $n
> are the arguments to be passed down.

Interesting approach. I don't think we need a delimiter though since we can just
do the following (main() modified to only pass along args that start with "a" or
"f"):

$ cat tst.sh
function sub {
printf '\tsub %s\n' "$@"
for subarg
do
printf '\t subarg = "%s"\n' "$subarg"
done
}

function main {
printf 'main %s\n' "$@"

for mainarg
do
printf ' mainarg = "%s"\n' "$mainarg"
case $mainarg in
[af]* ) set -- "$@" "$mainarg" ;;
* ) ;;
esac
shift
done

sub "$@"
}

main "a*" "b" "fgh" "x" "abc"
$
$ ./tst.sh
main a*
main b
main fgh
main x
main abc
mainarg = "a*"
mainarg = "b"
mainarg = "fgh"
mainarg = "x"
mainarg = "abc"
sub a*
sub fgh
sub abc
subarg = "a*"
subarg = "fgh"
subarg = "abc"

Any issues with that?

I think that's a nicer approach than using eval. Thanks!

Ed.

Ed Morton

unread,
Jan 22, 2012, 3:12:57 PM1/22/12
to
That's what I ended up with after seeing Kaz's suggestion so that's a good sign!

> BTW, it's the wrong way to use eval here. With eval, you'd want
> to do it:
>
> main() {
> c=1
> passargs=
> for i do
> if ...; then
> passargs="$passargs \"\${$c}\""
> fi
> c=$(($c + 1))
> done
> eval "sub $passargs"
> }
>
> (which is going to ask something like: cmd "${1}" "${4}" "${6}"
> to be evaluated which is harmless while cmd "some random strings
> which might contain $, ", \, `" is less so.

Got it, thanks.

Ed.

Kaz Kylheku

unread,
Jan 22, 2012, 3:43:44 PM1/22/12
to
On 2012-01-22, Ed Morton <morto...@gmail.com> wrote:
> Interesting approach. I don't think we need a delimiter though since we can just
> do the following (main() modified to only pass along args that start with "a" or
> "f"):
>
> $ cat tst.sh
> function sub {
> printf '\tsub %s\n' "$@"
> for subarg
> do
> printf '\t subarg = "%s"\n' "$subarg"
> done
> }
>
> function main {
> printf 'main %s\n' "$@"
>
> for mainarg
> do
> printf ' mainarg = "%s"\n' "$mainarg"
> case $mainarg in
> [af]* ) set -- "$@" "$mainarg" ;;
> * ) ;;
> esac
> shift
> done

Okay, this shows that the "for mainarg" only iterates over a copy
of the positional parameter which is captured at the time
the construct is entered, and does not see additional parameters
added via set.

So we know that if we have executed shift as many times as there
are iterations, we are left only with material that was added since.

Test with bash:

$ set -- 1 2 3
$ for x ; do echo $x ; set -- "$@" "$x" ; done
1
2
3
$ echo "$@"
1 2 3 1 2 3

I see.

Ed Morton

unread,
Jan 22, 2012, 4:12:03 PM1/22/12
to
Yeah, it was a good theory but it doesn't work when you add getopts and make it
a while loop which is my real situation. I was hoping that something like this
would work (modified again to pass along -a and -f opts and their args):

function sub {
printf '\tsub %s\n' "$@"
for subarg
do
printf '\t subarg="%s"\n' "$subarg"
done
}

function main {
printf 'main %s\n' "$@"

while getopts "a:f:bx" opt
do
printf '\t opt="%s", OPTARG="$OPTARG"\n' "$opt" "$OPTARG"
case $opt in
[af] ) set -- "$@" "-$opt" "$OPTARG" ;;
esac
done
shift "$(( OPTIND - 1 ))"

sub "$@"
}

main "-a*" "-b" "-fgh" "-x" "-abc"

$ ./tst.sh
main -a*
main -b
main -fgh
main -x
main -abc
opt="a", OPTARG="$OPTARG"
opt="*", OPTARG="$OPTARG"
opt="b", OPTARG="$OPTARG"
opt="", OPTARG="$OPTARG"
opt="f", OPTARG="$OPTARG"
opt="gh", OPTARG="$OPTARG"
opt="x", OPTARG="$OPTARG"
opt="", OPTARG="$OPTARG"
opt="a", OPTARG="$OPTARG"
opt="bc", OPTARG="$OPTARG"
....

but it just spins off into an endless loop. Not entirely unexpected but not sure
what to do about it!

Ed.

Kaz Kylheku

unread,
Jan 22, 2012, 5:25:01 PM1/22/12
to
On 2012-01-22, Ed Morton <morto...@gmail.com> wrote:
> On 1/22/2012 2:43 PM, Kaz Kylheku wrote:
>> Test with bash:
>>
>> $ set -- 1 2 3
>> $ for x ; do echo $x ; set -- "$@" "$x" ; done
>> 1
>> 2
>> 3
>> $ echo "$@"
>> 1 2 3 1 2 3
>>
>> I see.
>
> Yeah, it was a good theory but it doesn't work when you add getopts and make
> it a while loop which is my real situation. I was hoping that something like
> this would work (modified again to pass along -a and -f opts and their args):

In the real situation, you're not cleanly iterating over a copy of the
parameters, so the problem of termination comes back.

> but it just spins off into an endless loop. Not entirely unexpected but not
> sure what to do about it!

You need a termination test, and for that we can reintroduce the
sentinel value trick.

function sub {
printf '\tsub %s\n' "$@"
for subarg
do
printf '\t subarg="%s"\n' "$subarg"
done
}

function main {
printf 'main %s\n' "$@"

set -- "$@" "#EOA#"

while [ "$1" != "#EOA#" ] && getopts "a:f:bx" opt
do
printf '\t opt="%s", OPTARG="$OPTARG"\n' "$opt" "$OPTARG"
case $opt in
[af] ) set -- "$@" "-$opt" "$OPTARG" ;;
esac
done

while [ "$1" != "#EOA#" ] ; do
shift
done
shift

sub "$@"
}

main "$@"

$ ./main.sh "-a*" "-b" "-f 123"
main -a*
main -b
main -f 123
opt="a", OPTARG="$OPTARG"
opt="*", OPTARG="$OPTARG"
opt="b", OPTARG="$OPTARG"
opt="", OPTARG="$OPTARG"
opt="f", OPTARG="$OPTARG"
opt=" 123", OPTARG="$OPTARG"
sub -a
sub *
sub -f
sub 123
subarg="-a"
subarg="*"
subarg="-f"
subarg=" 123"

Ed Morton

unread,
Jan 22, 2012, 7:36:11 PM1/22/12
to
On 1/22/2012 4:25 PM, Kaz Kylheku wrote:
<snip>
I like the general approach but I don't like having to pick a specific string to
use as a terminator. How about if I count the number of real options first and
then stop the main loop when I've parsed that number of options:

function main {
printf 'main %s\n' "$@"

while getopts "a:f:bx" opt; do : ; done
numOpts="$OPTIND"

OPTIND=0
while [ "$OPTIND" != "$numOpts" ] && getopts "a:f:bx" opt
do
printf '\t opt="%s", OPTARG="$OPTARG"\n' "$opt" "$OPTARG"
case $opt in
[af] ) set -- "$@" "-$opt" "$OPTARG" ;;
esac
done
shift "$(( OPTIND - 1 ))"

sub "$@"
}

Still feels like I must be missing some simpler solution....

Regards,

Ed.

Ed Morton

unread,
Jan 22, 2012, 8:12:19 PM1/22/12
to
Hmm, I see that if there was something after the options that needs to be passed
to sub(), e.g. a list of file names as in:

main -a abc file1 file2

then that will end up before rather than after the options list passed to sub()
if I only do the above:

sub file1 file2 -a abc

so I'd need to do further processing to get that back to the end of the
arguments list.

I'm starting to think the eval approach is the better one after all:

$ cat tst.sh
function sub {
printf '\tsub %s\n' "$@"
for subarg
do
printf '\t subarg="%s"\n' "$subarg"
done
}

function main {
printf 'main %s\n' "$@"

passargs=""
while getopts "a:f:bx" opt
do
printf ' opt="-%s", OPTARG="%s"\n' "$opt" "$OPTARG"
case $opt in
[af] ) passargs="$passargs -$opt \"$OPTARG\"" ;;
esac
done
shift "$(( OPTIND - 1 ))"

eval "sub $passargs \"\$@\""
}

main -a "*" -b -f "gh ij" -x "file 1" "file 2"
$ ./tst.sh
main -a
main *
main -b
main -f
main gh ij
main -x
main file 1
main file 2
opt="-a", OPTARG="*"
opt="-b", OPTARG=""
opt="-f", OPTARG="gh ij"
opt="-x", OPTARG=""
sub -a
sub *
sub -f
sub gh ij
sub file 1
sub file 2
subarg="-a"
subarg="*"
subarg="-f"
subarg="gh ij"
subarg="file 1"
subarg="file 2"

Regards,

Ed.


Ed Morton

unread,
Jan 22, 2012, 8:15:37 PM1/22/12
to
On 1/22/2012 1:42 PM, Stephane Chazelas wrote:
<snip>
> main() {
> c=1
> passargs=
> for i do
> if ...; then
> passargs="$passargs \"\${$c}\""

Why would the above not just be:

passargs="$passargs \"$i\""

$i is the current argument when the "if" condition is tested, $c is the argument
at the position determined by the number of times the "if" condition has succeeded.

> fi
> c=$(($c + 1))
> done
> eval "sub $passargs"
> }

Regards,

Ed.

Ed Morton

unread,
Jan 22, 2012, 8:19:35 PM1/22/12
to
On 1/22/2012 7:15 PM, Ed Morton wrote:
> On 1/22/2012 1:42 PM, Stephane Chazelas wrote:
> <snip>
>> main() {
>> c=1
>> passargs=
>> for i do
>> if ...; then
>> passargs="$passargs \"\${$c}\""
>
> Why would the above not just be:
>
> passargs="$passargs \"$i\""
>
> $i is the current argument when the "if" condition is tested, $c is the argument
> at the position determined by the number of times the "if" condition has succeeded.

Ignore that last sentence, don't know what I was thinking. I still don't see why
$i isn't enough though.

Ed

Kaz Kylheku

unread,
Jan 22, 2012, 9:05:35 PM1/22/12
to
On 2012-01-23, Ed Morton <morto...@gmail.com> wrote:
> printf '\t opt="%s", OPTARG="$OPTARG"\n' "$opt" "$OPTARG"

This printf is causing confusing output! Because you have only a single %s,
the printf is iterating over the "$opt" and "$OPTARG".

You clearly this:

> printf '\t opt="%s", OPTARG="%s"\n' "$opt" "$OPTARG"

You have to capture the total number of arguments, then make sure you
rotate through all of them. Also, you have to terminate the option
processing loop once OPTIND hits the original total:

function main {
printf 'main %s\n' "$@"

numparams=$#

while [ $OPTIND -le $numparams ] && getopts "a:f:bx" opt
do
printf '\t opt="%s", OPTARG="%s"\n' "$opt" "$OPTARG"
case $opt in
[af] ) set -- "$@" "-$opt" "$OPTARG" ;;
esac
done

shift $(( OPTIND - 1 ))

i=$OPTIND # can we just use OPTIND in place of i?

# now keep going! consume up to $numparams from front, rotate to back:
while [ $i -le $numparams ] ; do
set -- "$@" "$1"
shift
i=$(( i + 1 ))
done

sub "$@"
}

$ ./main.sh -a arg -b -f s foo bar xyzzy
main -a
main arg
main -b
main -f
main s
main foo
main bar
main xyzzy
opt="a", OPTARG="arg"
opt="b", OPTARG=""
opt="f", OPTARG="s"
sub -a
sub arg
sub -f
sub s
sub foo
sub bar
sub xyzzy
subarg="-a"
subarg="arg"
subarg="-f"
subarg="s"
subarg="foo"
subarg="bar"
subarg="xyzzy"

Ed Morton

unread,
Jan 22, 2012, 9:27:46 PM1/22/12
to
On 1/22/2012 8:05 PM, Kaz Kylheku wrote:
> On 2012-01-23, Ed Morton<morto...@gmail.com> wrote:
>> printf '\t opt="%s", OPTARG="$OPTARG"\n' "$opt" "$OPTARG"
>
> This printf is causing confusing output! Because you have only a single %s,
> the printf is iterating over the "$opt" and "$OPTARG".
>
> You clearly this:
>
>> printf '\t opt="%s", OPTARG="%s"\n' "$opt" "$OPTARG"
>
> You have to capture the total number of arguments, then make sure you
> rotate through all of them.

Yeah, that was just a sloppy bug. I corrected it in my followup post.

Also, you have to terminate the option
> processing loop once OPTIND hits the original total:
>
> function main {
> printf 'main %s\n' "$@"
>
> numparams=$#

I don't think that's correct. You need to do a getopts loop to determine how
many options are contained in the argument list since getopts will split "-bx"
(one argument) into "-b" "-x" (two options) and you're going to compare $OPTIND
with $numparams.

>
> while [ $OPTIND -le $numparams ]&& getopts "a:f:bx" opt
> do
> printf '\t opt="%s", OPTARG="%s"\n' "$opt" "$OPTARG"
> case $opt in
> [af] ) set -- "$@" "-$opt" "$OPTARG" ;;
> esac
> done
>
> shift $(( OPTIND - 1 ))
>
> i=$OPTIND # can we just use OPTIND in place of i?
>
> # now keep going! consume up to $numparams from front, rotate to back:
> while [ $i -le $numparams ] ; do
> set -- "$@" "$1"
> shift
> i=$(( i + 1 ))
> done
>
> sub "$@"
> }

Just seems like it's getting to be a lot more code and more complex code
compared to the eval method for no benefit that I can see.

Ed.

Kaz Kylheku

unread,
Jan 22, 2012, 10:06:54 PM1/22/12
to
On 2012-01-23, Ed Morton <morto...@gmail.com> wrote:
> On 1/22/2012 8:05 PM, Kaz Kylheku wrote:
>> On 2012-01-23, Ed Morton<morto...@gmail.com> wrote:
>>> printf '\t opt="%s", OPTARG="$OPTARG"\n' "$opt" "$OPTARG"
>>
>> This printf is causing confusing output! Because you have only a single %s,
>> the printf is iterating over the "$opt" and "$OPTARG".
>>
>> You clearly this:
>>
>>> printf '\t opt="%s", OPTARG="%s"\n' "$opt" "$OPTARG"
>>
>> You have to capture the total number of arguments, then make sure you
>> rotate through all of them.
>
> Yeah, that was just a sloppy bug. I corrected it in my followup post.
>
> Also, you have to terminate the option
>> processing loop once OPTIND hits the original total:
>>
>> function main {
>> printf 'main %s\n' "$@"
>>
>> numparams=$#
>
> I don't think that's correct. You need to do a getopts loop to determine how
> many options are contained in the argument list since getopts will split "-bx"
> (one argument) into "-b" "-x" (two options) and you're going to compare $OPTIND
> with $numparams.

But $OPTIND counts params, not options. We just have to remember how many
positional parameters there originally were, so that after getopts is done
telling us how many /it/ consumed, we can consume the rest (put them at the
tail).

The invariant is that we process all the arguments.

-bx works fine:

$ ./main.sh -a arg -bx -f s foo bar xyzzy
main -a
main arg
main -bx
main -f
main s
main foo
main bar
main xyzzy
opt="a", OPTARG="arg"
opt="b", OPTARG=""
opt="x", OPTARG=""
opt="f", OPTARG="s"
sub -a
sub arg
sub -f
sub s
sub foo
sub bar
sub xyzzy
subarg="-a"
subarg="arg"
subarg="-f"
subarg="s"
subarg="foo"
subarg="bar"
subarg="xyzzy"


Various corner cases:

$ ./main.sh
main
sub
$ ./main.sh asdf
main asdf
sub asdf
subarg="asdf"
$ ./main.sh -b asdf
main -b
main asdf
opt="b", OPTARG=""
sub asdf
subarg="asdf"
$ ./main.sh -a d asdf
main -a
main d
main asdf
opt="a", OPTARG="d"
sub -a
sub d
sub asdf
subarg="-a"
subarg="d"
subarg="asdf"

Kaz Kylheku

unread,
Jan 22, 2012, 10:40:46 PM1/22/12
to
On 2012-01-23, Ed Morton <morto...@gmail.com> wrote:
> On 1/22/2012 7:15 PM, Ed Morton wrote:
>> On 1/22/2012 1:42 PM, Stephane Chazelas wrote:
>> <snip>
>>> main() {
>>> c=1
>>> passargs=
>>> for i do
>>> if ...; then
>>> passargs="$passargs \"\${$c}\""
>>
>> Why would the above not just be:
>>
>> passargs="$passargs \"$i\""
>>
>> $i is the current argument when the "if" condition is tested, $c is the argument
>> at the position determined by the number of times the "if" condition has succeeded.
>
> Ignore that last sentence, don't know what I was thinking. I still don't see why
> $i isn't enough though.

Because then you bring in escaping issues (which is the big reason to
avoid eval in the first place).

What if i contains stuff like this:

$TERM
a"

The stuff going into eval will look like this:

"$TERM" a" # unwanted expansion and syntax error!

Stephane did this the smart way: generating code which references
the positional parameters, rather than generating code in which
their text is already interpolated.

The \" \" quotes are there just to prevent word splitting.

The code going into eval looks like

"${1}" "${2}" "${3}" ... # refs to the parameters, not the text itself!

Of course, the parameters he doesn't want are omitted.

The curly braces are overkill, by the way. This should be enough,
but maybe it's less readable:

# generate "$1" "$2" "$3" ...

passargs="$passargs \"\$$c\""

This is like the difference between these:

# computed variable name
eval "$VAR=\$VAL" # evaled code: FOO=$VAL # good!

# versus
eval "$VAR=\"$VAL\"" # evaled code: FOO="<unsafe stuff>" # bad!

I don't think the trick applies here very easily because your output
parameter list does not necessarily consist of items which are parameters in
the original list. Two parameters -af 3 become three parameters -a -f 3.

You could solve this problem by generating variables:

gencounter=$((counter + 1))
genvar=GEN$gencounter
eval GEN$counter=\$value # $value holds datum we want.
passargs="$passargs \"\$GEN$gencounter\""

now the evaled code ends up being

"$GEN0" "$GEN1" "$GEN2" ...

The only problem with this is pollution.


The working code is not any simpler in the end.

The first cut doesn't quite work:

function main {
printf 'main %s\n' "$@"

passargs=
gc=0

while getopts "a:f:bx" opt
do
printf '\t opt="%s", OPTARG="%s"\n' "$opt" "$OPTARG"
case $opt in
[af] ) eval "GEN$gc=-$opt; GEN$((gc + 1))=\$OPTARG"
passargs="$passargs \"\$GEN$gc\" \"\$GEN$((gc + 1))\""
gc=$(( gc + 2 ))
esac
done

shift $(( OPTIND - 1 ))

eval sub "$passargs" "$@"
}

The problem is with the "$@" in the eval. Watch what happens with $TERM:

$ ./main.sh -f d -a d\" '$TERM' asdf
main -f
main d
main -a
main d"
main $TERM
main asdf
opt="f", OPTARG="d"
opt="a", OPTARG="d""
sub -f
sub d
sub -a
sub d"
sub xterm
sub asdf
subarg="-f"
subarg="d"
subarg="-a"
subarg="d""
subarg="xterm"
subarg="asdf"

Oops! To fix that we need a second loop where we add more stuff to passargs:

function main {
printf 'main %s\n' "$@"

passargs=
gc=0

while getopts "a:f:bx" opt
do
printf '\t opt="%s", OPTARG="%s"\n' "$opt" "$OPTARG"
case $opt in
[af] ) eval "GEN$gc=-$opt; GEN$((gc + 1))=\$OPTARG"
passargs="$passargs \"\$GEN$gc\" \"\$GEN$((gc + 1))\""
gc=$(( gc + 2 ))
esac
done

shift $(( OPTIND - 1 ))

i=1
for remaining ; do
passargs="$passargs \"\$$i\""
i=$((i + 1))
done

printf 'what is being evaled: %s\n' "$passargs"

eval sub "$passargs"
}

$ ./main.sh -f d -a d\" '$TERM' asdf
main -f
main d
main -a
main d"
main $TERM
main asdf
opt="f", OPTARG="d"
opt="a", OPTARG="d""
what is being evaled: "$GEN0" "$GEN1" "$GEN2" "$GEN3" "$1" "$2"
sub -f
sub d
sub -a
sub d"
sub $TERM
sub asdf
subarg="-f"
subarg="d"
subarg="-a"
subarg="d""
subarg="$TERM"
subarg="asdf"

Oh man, gensyms in shell code, *slap forehead*.

Make my latte de-eval-inated, please ...

Seebs

unread,
Jan 23, 2012, 2:58:19 AM1/23/12
to
> Is there a better (simpler and/or more secure and/or whatever you think "better"
> means) way to pass main()s arguments as-is down to sub() than using eval?

You can at least change the way eval is used.

arg_counter=0
for arg; do
if $condition; then
eval arg_$arg_counter=\$arg
arg_counter=$(expr $arg_counter + 1)
fi
done
arg_used=0
set --
while [ $arg_used < $arg_counter ]; do
eval this_arg=\$arg_$arg_used
set -- "$@" "$this_arg"
arg_used=$(expr $arg_used + 1)
done
subroutine "$@"

It's not fast, it's not pretty, but it does work and is robust.

(modulos typos)

-s
--
Copyright 2011, all wrongs reversed. Peter Seebach / usenet...@seebs.net
http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
I am not speaking for my employer, although they do rent some of my opinions.

Martin Vaeth

unread,
Jan 23, 2012, 3:09:33 AM1/23/12
to
Ed Morton <morto...@gmail.com> wrote:
> function sub {

The POSIX syntax for this is "sub() {"
If you have another shell, you can simply use an array
for the arguments.

> passargs="$passargs \"$mainarg\""

This is dangerous. If your args do not contain ' you can use
passargs="$passargs '$mainarg'"
and if you args do contain ' you have to substitute those
in $mainarg first by '\'' (which is tricky in POSIX).

Then the eval is safe. I would be careful and use also set -f
before the eval, although this should not be necessary.

Stephane Chazelas

unread,
Jan 23, 2012, 6:31:57 AM1/23/12
to
2012-01-23 03:40:46 +0000, Kaz Kylheku:
[...]
> The code going into eval looks like
>
> "${1}" "${2}" "${3}" ... # refs to the parameters, not the text itself!
>
> Of course, the parameters he doesn't want are omitted.
>
> The curly braces are overkill, by the way. This should be enough,
> but maybe it's less readable:
>
> # generate "$1" "$2" "$3" ...
[...]

No. Except with ash and zsh (when not in sh or ksh emulation
mode), you need the curly braces for ${10} and above (Bourne,
bash and all implementations of ksh interpret $10 as ${1}0).

--
Stephane

Ed Morton

unread,
Jan 23, 2012, 8:39:34 AM1/23/12
to
On 1/22/2012 9:06 PM, Kaz Kylheku wrote:
> On 2012-01-23, Ed Morton<morto...@gmail.com> wrote:
>> On 1/22/2012 8:05 PM, Kaz Kylheku wrote:
>>> On 2012-01-23, Ed Morton<morto...@gmail.com> wrote:
>>>> printf '\t opt="%s", OPTARG="$OPTARG"\n' "$opt" "$OPTARG"
>>>
>>> This printf is causing confusing output! Because you have only a single %s,
>>> the printf is iterating over the "$opt" and "$OPTARG".
>>>
>>> You clearly this:
>>>
>>>> printf '\t opt="%s", OPTARG="%s"\n' "$opt" "$OPTARG"
>>>
>>> You have to capture the total number of arguments, then make sure you
>>> rotate through all of them.
>>
>> Yeah, that was just a sloppy bug. I corrected it in my followup post.
>>
>> Also, you have to terminate the option
>>> processing loop once OPTIND hits the original total:
>>>
>>> function main {
>>> printf 'main %s\n' "$@"
>>>
>>> numparams=$#
>>
>> I don't think that's correct. You need to do a getopts loop to determine how
>> many options are contained in the argument list since getopts will split "-bx"
>> (one argument) into "-b" "-x" (two options) and you're going to compare $OPTIND
>> with $numparams.
>
> But $OPTIND counts params, not options.

Yup, you're right. Thanks,

Ed.

Dave Gibson

unread,
Jan 23, 2012, 9:00:33 AM1/23/12
to
Ed Morton <morto...@gmail.com> wrote:

> Hmm, I see that if there was something after the options that needs
> to be passed to sub(), e.g. a list of file names as in:
>
> main -a abc file1 file2
>
> then that will end up before rather than after the options list
> passed to sub() if I only do the above:
>
> sub file1 file2 -a abc
>
> so I'd need to do further processing to get that back to the end of
> the arguments list.
>
> I'm starting to think the eval approach is the better one after all:

Use an array.

>
> $ cat tst.sh
> function sub {
> printf '\tsub %s\n' "$@"
> for subarg
> do
> printf '\t subarg="%s"\n' "$subarg"
> done
> }
>
> function main {
> printf 'main %s\n' "$@"
>
> passargs=""

unset passargs

> while getopts "a:f:bx" opt
> do
> printf ' opt="-%s", OPTARG="%s"\n' "$opt" "$OPTARG"
> case $opt in
> [af] ) passargs="$passargs -$opt \"$OPTARG\"" ;;

[af] ) passargs+=( "-$opt" "$OPTARG" ) ;;

> esac
> done
> shift "$(( OPTIND - 1 ))"
>
> eval "sub $passargs \"\$@\""

sub "${passargs[@]}" "$@"

Ed Morton

unread,
Jan 23, 2012, 9:20:21 AM1/23/12
to
Got it. Then I think I can just use an array indexed by OPTIND instead of $1,
$2, etc:

$ cat tst.sh
sub() {
printf '\tsub %s\n' "$@"
for subarg
do
printf '\t subarg="%s"\n' "$subarg"
done
}

main() {
printf 'main %s\n' "$@"

passargs=""
while getopts "a:f:bx" opt
do
printf ' opt="-%s", OPTARG="%s", OPTIND="%s"\n' "$opt" "$OPTARG"
"$OPTIND"

case $opt in
[af] ) optArgs[$OPTIND]="$OPTARG"
passargs="$passargs -$opt \"\${optArgs[$OPTIND]}\""
;;
esac
done
shift "$(( OPTIND - 1 ))"

eval "sub $passargs \"\$@\""
}

main -a"*" -bx -f "gh ij" "file 1" "file 2"
$ ./tst.sh
main -a*
main -bx
main -f
main gh ij
main file 1
main file 2
opt="-a", OPTARG="*", OPTIND="2"
opt="-b", OPTARG="", OPTIND="2"
opt="-x", OPTARG="", OPTIND="3"
opt="-f", OPTARG="gh ij", OPTIND="5"
sub -a
sub *
sub -f
sub gh ij
sub file 1
sub file 2
subarg="-a"
subarg="*"
subarg="-f"
subarg="gh ij"
subarg="file 1"
subarg="file 2"
<snip>
> Oh man, gensyms in shell code, *slap forehead*.

Could you expand on that?

I thought of a problem with the "set -- ..." version of the code by the way -
sometimes you need to pass different arguments to different sub-scripts, e.g.:

$ cat ./tst.sh
sub1() {
printf '\tsub1 %s\n' "$@"
for subarg
do
printf '\t subarg="%s"\n' "$subarg"
done
}

sub2() {
printf '\tsub2 %s\n' "$@"
for subarg
do
printf '\t subarg="%s"\n' "$subarg"
done
}

main() {
printf 'main %s\n' "$@"

sub1args=""
sub2args=""
while getopts "a:f:p:bx" opt
do
printf ' opt="-%s", OPTARG="%s", OPTIND="%s"\n' "$opt" "$OPTARG"
"$OPTIND"

case $opt in
[af] ) optArgs[$OPTIND]="$OPTARG"
sub1args="$sub1args -$opt \"\${optArgs[$OPTIND]}\""
;;
p ) optArgs[$OPTIND]="$OPTARG"
sub2args="$sub2args -$opt \"\${optArgs[$OPTIND]}\""
;;
esac
done
shift "$(( OPTIND - 1 ))"

eval "sub1 $sub1args \"\$1\""
eval "sub2 $sub2args \"\$2\""
}

main -a"*" -bx -p "sub2info" -f "gh ij" "file 1" "file 2"
$ ./tst.sh
main -a*
main -bx
main -p
main sub2info
main -f
main gh ij
main file 1
main file 2
opt="-a", OPTARG="*", OPTIND="2"
opt="-b", OPTARG="", OPTIND="2"
opt="-x", OPTARG="", OPTIND="3"
opt="-p", OPTARG="sub2info", OPTIND="5"
opt="-f", OPTARG="gh ij", OPTIND="7"
sub1 -a
sub1 *
sub1 -f
sub1 gh ij
sub1 file 1
subarg="-a"
subarg="*"
subarg="-f"
subarg="gh ij"
subarg="file 1"
sub2 -p
sub2 sub2info
sub2 file 2
subarg="-p"
subarg="sub2info"
subarg="file 2"

AFAIK you cant do that if you're relying on the expansion of "$@" to contain all
your arguments.

Ed.

Ed Morton

unread,
Jan 23, 2012, 9:22:02 AM1/23/12
to
On 1/23/2012 1:58 AM, Seebs wrote:
>> Is there a better (simpler and/or more secure and/or whatever you think "better"
>> means) way to pass main()s arguments as-is down to sub() than using eval?
>
> You can at least change the way eval is used.
>
> arg_counter=0
> for arg; do
> if $condition; then
> eval arg_$arg_counter=\$arg
> arg_counter=$(expr $arg_counter + 1)
> fi
> done
> arg_used=0
> set --
> while [ $arg_used< $arg_counter ]; do
> eval this_arg=\$arg_$arg_used
> set -- "$@" "$this_arg"
> arg_used=$(expr $arg_used + 1)
> done
> subroutine "$@"
>
> It's not fast, it's not pretty, but it does work and is robust.
>
> (modulos typos)
>
> -s

Got it. I think I can do the same with an array indexed by $OPTIND though rather
than a variable that contains a new counter and a second eval. See my recent
post elsethread.

Thanks,

Ed.

Stephane Chazelas

unread,
Jan 23, 2012, 9:29:29 AM1/23/12
to
2012-01-23 08:20:21 -0600, Ed Morton:
[...]
> [af] ) optArgs[$OPTIND]="$OPTARG"
> passargs="$passargs -$opt \"\${optArgs[$OPTIND]}\""
[...]
> eval "sub $passargs \"\$@\""
[...]

If you're using arrays already, why are you using a scalar for
$passargs?

passargs+=("-$opt" "$OPTARG")
[...]
sub "$passargs[@]"

(zsh syntax).

--
Stephane

Ed Morton

unread,
Jan 23, 2012, 9:33:23 AM1/23/12
to
And we have a winner!

$ cat tst.sh
sub1() {
printf '\tsub1 %s\n' "$@"
for subarg
do
printf '\t subarg="%s"\n' "$subarg"
done
}

sub2() {
printf '\tsub2 %s\n' "$@"
for subarg
do
printf '\t subarg="%s"\n' "$subarg"
done
}

main() {
printf 'main %s\n' "$@"

unset sub1args
unset sub2args
while getopts "a:f:p:bx" opt
do
printf ' opt="-%s", OPTARG="%s"\n' "$opt" "$OPTARG"

case $opt in
[af] ) sub1args+=( "-$opt" "$OPTARG" ) ;;
p ) sub2args+=( "-$opt" "$OPTARG" ) ;;
esac
done
shift "$(( OPTIND - 1 ))"

sub1 "${sub1args[@]}" "$1"
sub2 "${sub2args[@]}" "$2"
}

main -a"*" -bx -p 'sub2"
info' -f "gh ij" "file 1" "file 2"
$ ./tst.sh
main -a*
main -bx
main -p
main sub2"
info
main -f
main gh ij
main file 1
main file 2
opt="-a", OPTARG="*"
opt="-b", OPTARG=""
opt="-x", OPTARG=""
opt="-p", OPTARG="sub2"
info"
opt="-f", OPTARG="gh ij"
sub1 -a
sub1 *
sub1 -f
sub1 gh ij
sub1 file 1
subarg="-a"
subarg="*"
subarg="-f"
subarg="gh ij"
subarg="file 1"
sub2 -p
sub2 sub2"
info
sub2 file 2
subarg="-p"
subarg="sub2"
info"
subarg="file 2"

Simple and elegant. Love it. Any caveats?

Thanks,

Ed.

Ed Morton

unread,
Jan 23, 2012, 9:36:46 AM1/23/12
to
Never occurred to me. I don't do a lot of actual PROGRAMMING in shell, I usually
just call tools from it. Usually my shell scripts are a couple of case
statements and/or a pipe!

Thanks,

Ed.

Ed Morton

unread,
Jan 23, 2012, 9:54:18 AM1/23/12
to
On 1/23/2012 8:29 AM, Stephane Chazelas wrote:
In case anyone cares, I found with bash I need to add curly braces:

sub "${passargs[@]}"

I assume that'll work in zsh too.

Thanks again,

Ed.

Ed Morton

unread,
Jan 23, 2012, 9:58:39 AM1/23/12
to
Thanks, I think the recently posted array solution is the one I'll go with.

Ed.

Kaz Kylheku

unread,
Jan 23, 2012, 10:06:40 AM1/23/12
to
Oh sheesh! Minimal munch principle for tokenization. :)

Dave Gibson

unread,
Jan 24, 2012, 12:02:06 PM1/24/12
to
Ed Morton <morto...@gmail.com> wrote:

[ shell arrays ]

> Simple and elegant. Love it. Any caveats?

Minor portability issues. Mostly whether arrays are indexed from 0
(bash, ksh93, mksh) or 1 (zsh).

Martin Vaeth

unread,
Jan 25, 2012, 11:51:15 AM1/25/12
to
Dave Gibson <dave.gma...@googlemail.com.invalid> wrote:
> Ed Morton <morto...@gmail.com> wrote:
>
> [ shell arrays ]
>
>> Simple and elegant. Love it. Any caveats?
>
> Minor portability issues.

Minor?

> Mostly whether arrays are indexed from 0

Mostly whether arrays are supported at all:
Need not be the case even for POSIX compatible shells.
Is not the case for dash (system shell on many linux systems),
busybox (system shell on many rescue and boot systems like
linux initramfs), and as I suppose for many other (all?)
Bourne, Sun, *BSD system shells.

If you want to rely on particular shell features,
it is probably the best idea to use zsh which has
the most intuitive treatment of arrays, variables,
and quoting (and also in almost all other aspects
the most advanced features) and in contrast to some
other shells with feature extensions has only very
rarely backwards compatibility issues.

Dave Gibson

unread,
Jan 25, 2012, 4:40:36 PM1/25/12
to
Martin Vaeth <va...@mathematik.uni-wuerzburg.de> wrote:
> Dave Gibson <dave.gma...@googlemail.com.invalid> wrote:
>> Ed Morton <morto...@gmail.com> wrote:
>>
>> [ shell arrays ]
>>
>>> Simple and elegant. Love it. Any caveats?
>>
>> Minor portability issues.
>
> Minor?
>
>> Mostly whether arrays are indexed from 0

This is relevant:

>> (bash, ksh93, mksh) or 1 (zsh).

>
> Mostly whether arrays are supported at all:

I was referring to the differences between array implementations
in the named shells -- specific rather than general portability.

Dave Gibson

unread,
Jan 28, 2012, 4:04:45 PM1/28/12
to
Dave Gibson <dave.gma...@googlemail.com.invalid> wrote:
> Martin Vaeth <va...@mathematik.uni-wuerzburg.de> wrote:
>> Dave Gibson <dave.gma...@googlemail.com.invalid> wrote:
>>> Ed Morton <morto...@gmail.com> wrote:
>>>
>>> [ shell arrays ]
>>>
>>>> Simple and elegant. Love it. Any caveats?
>>>
>>> Minor portability issues.
>>
>> Minor?
>>
>>> Mostly whether arrays are indexed from 0
>
> This is relevant:
>
>>> (bash, ksh93, mksh) or 1 (zsh).
>
>>
>> Mostly whether arrays are supported at all:
>
> I was referring to the differences between array implementations
> in the named shells -- specific rather than general portability.

You missed one, you moron.
0 new messages