How do I pass an array of strings to a Bash script and join that array?

[dontsende...@gmail.com]

I want to join an array of strings on the string "%2C+". My shell script launch looks like this: https://stackoverflow.com/questions/50033402/how-do-i-pass-an-array-of-strings-to-a-bash-script-and-join-that-array

[Barry Margolin]

In article <d6ca9e9f-f1d5-4b1b...@googlegroups.com>,

From your SO post:

>./download-data $("state_code" "county_code")

$(...) doesn't create an array, it executes the command and substitutes
the output back into the command line. So this is trying to execute the
state_code command, which doesn't exist.

To create an array you do:

arrayname=("state_code" "county_code")

Note that there's no $ before ().

To pass the array to a script, do:

./download_data "${arrayname[@]}"

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Helmut Waitzmann]

dontsende...@gmail.com:

> I want to join an array of strings on the string "%2C+". My
> shell script launch looks like this:

Quoting and reformatting
> https://stackoverflow.com/questions/50033402/how-do-i-pass-an-array-of-strings-to-a-bash-script-and-join-that-array

> But when I run
>
> ./download-data $("state_code" "county_code"), I get this error
> in the terminal: bash: state_code: command not found.
>
> I need to pass the argument as an array since I plan to pass
> more arrays later on.
> Something like
> ./download-data $("state_code" "county_code") $("more" "string").

If the shell script “./download-data” as well as the invoking
shell is a bash, you can do:

* In the shell script “./download-data”:

n=0 &&
while ${arg+:} false
do
n=$((n+1))
eval "array_${n}=( $arg )"
shift
done

* Invoke the shell script “./download-data” by

./donwload-data \
"$( printf '%q ' "state_code" "county_code" )"
"$( printf '%q ' "more" "string" )"

When invoked like this, the array “array_1” will
consist of the two elements "state_code" and "county_code",
while the array "array_2" will consist of the two elements
"more" and "string".

(If at least one of the two involved shells is not a bash, it
gets more complicated.)

[Janis Papanagnou]

On 26.04.2018 23:05, Helmut Waitzmann wrote:
> dontsende...@gmail.com:
>
>> I want to join an array of strings on the string "%2C+". My
>> shell script launch looks like this:
>
> Quoting and reformatting
>> https://stackoverflow.com/questions/50033402/how-do-i-pass-an-array-of-strings-to-a-bash-script-and-join-that-array
>
>> But when I run
>>
>> ./download-data $("state_code" "county_code"), I get this error
>> in the terminal: bash: state_code: command not found.
>>
>> I need to pass the argument as an array since I plan to pass
>> more arrays later on.
>> Something like
>> ./download-data $("state_code" "county_code") $("more" "string").
>
> If the shell script “./download-data” as well as the invoking
> shell is a bash, you can do:
>
> * In the shell script “./download-data”:
>
> n=0 &&
> while ${arg+:} false
> do
> n=$((n+1))
> eval "array_${n}=( $arg )"
> shift

What are you shifting here? (Is $arg maybe supposed to be $1 ?)

> done
>
> * Invoke the shell script “./download-data” by
>
> ./donwload-data \
> "$( printf '%q ' "state_code" "county_code" )"
> "$( printf '%q ' "more" "string" )"
>
> When invoked like this, the array “array_1” will
> consist of the two elements "state_code" and "county_code",
> while the array "array_2" will consist of the two elements
> "more" and "string".
>
> (If at least one of the two involved shells is not a bash, it
> gets more complicated.)

I don't see any bash-specifics in your code and would expect that
it works also in ksh and zsh.

Janis

[Helmut Waitzmann]

Janis Papanagnou <janis_pa...@hotmail.com>:

> On 26.04.2018 23:05, Helmut Waitzmann wrote:

>> If the shell script “./download-data” as well as the invoking
>> shell is a bash, you can do:
>>
>> * In the shell script “./download-data”:
>>
>> n=0 &&
>> while ${arg+:} false
>> do
>> n=$((n+1))
>> eval "array_${n}=( $arg )"
>> shift
>
> What are you shifting here? (Is $arg maybe supposed to be $1 ?)

Oh sorry. Yes, you are totally correct: It should have been

n=0 &&
while ${1+:} false
do
n=$((n+1))
eval "array_${n}=( $1 )"
shift

(I first tried to use "for" rather than "while", using the
iteration variable "arg", then changed the code and missed to
replace "arg" by "1".)

[Helmut Waitzmann]

Janis Papanagnou <janis_pa...@hotmail.com>:

> On 26.04.2018 23:05, Helmut Waitzmann wrote:

> I don't see any bash-specifics in your code and would expect that
> it works also in ksh and zsh.

I don't know whether ksh and zsh support arrays and a built-in
"printf" using a "%q" format specifier in the same way like bash
does.

If not, it might not work to invoke the bash script from a ksh or
zsh or vice versa.

In a POSIX-only conformant system, the shell doesn't know arrays
and the "printf" utility doesn't know the "%q" format specifier.

(The problem can be solved by relying on POSIX only, too, but,
depending on the exact use case, that may be much more
complicated.)

[Janis Papanagnou]

On 27.04.2018 10:31, Helmut Waitzmann wrote:
> Janis Papanagnou <janis_pa...@hotmail.com>:
>> On 26.04.2018 23:05, Helmut Waitzmann wrote:
>
>> I don't see any bash-specifics in your code and would expect that
>> it works also in ksh and zsh.
>
> I don't know whether ksh and zsh support arrays and a built-in
> "printf" using a "%q" format specifier in the same way like bash
> does.

Most "bash specific" extensions are inherited, and mostly from ksh
(which still offers a lot more unique features to inherit or not).

Specifically arrays are very old, and...
<hard context switch>
... I stumbled across a weird ksh behaviour here:

unset a
echo "${#a[@]}" "${a[@]}"
a=( )
echo "${#a[@]}" "${a[@]}"
a+=( aaa )
echo "${#a[@]}" "${a[@]}"
a+=( bbb )
echo "${#a[@]}" "${a[@]}"

# ksh output
0
1 (
)
1 aaa
2 aaa bbb

# bash and zsh output
0
0
1 aaa
2 aaa bbb


Janis

> [...]

[Dave Sines]

Janis Papanagnou <janis_pa...@hotmail.com> wrote:
[...]

> <hard context switch>
> ... I stumbled across a weird ksh behaviour here:
>
> unset a
> echo "${#a[@]}" "${a[@]}"
> a=( )

[...]

> # ksh output
> 0
> 1 (
> )

$a is a compound variable.

As a work-around either use an unset variable in the assignment

unset a
a=( $a )

or explicitly declare a to be an array

typeset -a a=( )

[Janis Papanagnou]

On 28.04.2018 14:17, Dave Sines wrote:
> Janis Papanagnou <janis_pa...@hotmail.com> wrote:
> [...]
>> <hard context switch>
>> ... I stumbled across a weird ksh behaviour here:
>>
>> unset a
>> echo "${#a[@]}" "${a[@]}"
>> a=( )
> [...]
>> # ksh output
>> 0
>> 1 (
>> )
>
> $a is a compound variable.

Hmm.. - I haven't thought of that. I'd rather have expected that a
compound variable would have needed an explicit declaration and not
the more classical array. Learned something new.

> As a work-around either use an unset variable in the assignment
>
> unset a
> a=( $a )
>
> or explicitly declare a to be an array
>
> typeset -a a=( )

It seems that declaring it separately as

typeset -a a
a=( )

doesn't work. Sadly zsh has also problems with

typeset -a a=( )

The unset a ; a=( $a ) work-around seems to work in ksh, bash, zsh.

Thanks!

Janis

[Janis Papanagnou]

On 28.04.2018 14:43, Janis Papanagnou wrote:
>> [...]

>
> It seems that declaring it separately as
>
> typeset -a a
> a=( )
>
> doesn't work. Sadly zsh has also problems with
>
> typeset -a a=( )

I notice that a simple typeset -a a prevents the zsh issue with the
assignment, and omitting an explicit separate assignment a=( ) avoids
the ksh issue.

[Martijn Dekker]

Op 27-04-18 om 09:07 schreef Helmut Waitzmann:
> while ${1+:} false

I hadn't seen that trick before. Clever, but non-obvious. To understand
what it does, anyone reading the code requires a keen understanding of
how the shell's parameter substitution and empty removal mechanisms work
-- including that the command name is just another argument and
therefore subject to empty removal, so that the next argument becomes
the command name instead.

The following works the same but is rather easier to understand:

while [ "$#" -gt 0 ]

- M.

[Stephane Chazelas]

2018-05-02 21:37:16 +0100, Martijn Dekker:

> Op 27-04-18 om 09:07 schreef Helmut Waitzmann:
> > while ${1+:} false
>
> I hadn't seen that trick before.

[...]

This kind of thing is sometimes seen in things like:

# save IFS
oldIFS=$IFS; ${IFS+:} unset oldIFS
case $- in
*f*) restore_glob=;;
*) set -f; restore_glob='set +f'
esac

IFS=:
set -- $PATH'' # split on :

# restore IFS
IFS=$oldIFS; ${oldIFS+:} unset IFS
eval "$restore_glob"

It's important as an empty $IFS has a very different meaning
from an unset $IFS.

The "more legible" version of ${IFS+:} unset oldIFS would be:

if [ "${IFS+set}" != set ]; then
unset oldIFS
fi

which is only marginally more legible and significantly longer
(for something that is already quite long for such a simple
thing to do).

Of course modern shells like zsh have more adequate ways to
split strings.

--
Stephane

[Helmut Waitzmann]

Martijn Dekker <mar...@inlv.demon.nl>:

> Op 27-04-18 om 09:07 schreef Helmut Waitzmann:

>> while ${1+:} false
>
> I hadn't seen that trick before. Clever, but non-obvious. To
> understand what it does, anyone reading the code requires a keen
> understanding of how the shell's parameter substitution and empty
> removal mechanisms work -- including that the command name is just
> another argument and therefore subject to empty removal, so that the
> next argument becomes the command name instead.

You are totally right. I should have added a comment.

>
> The following works the same but is rather easier to understand:
>
> while [ "$#" -gt 0 ]

Yes, of course. Because

>> while ${1+:} false

was a change from my incorrect (see some posts before)

“while ${arg+:} false”,

I didn't bother to replace it by

“while [ "$#" -gt 0 ]”,

which is an invocation of the (maybe non‐built‐in) “[” utility.


Let me explain my motivation for

“${arg+:} false”:

As Stephane Chazelas wrote, an equivalent command for

“${arg+:} false”

using the “[” utility would be

“[ "${arg+set}" = set ]”.

Another one would be

“[ -n "${arg+set}" ]”.

A third one would be

“[ ${arg+set} ]”.

All of these alternatives use the “${arg+...}” mechanism of the
shell; and it turns out, that the core logic of these alternatives
(i.e. to investigate whether the parameter “arg” is set or unset)
is achieved by the “${arg+...}” mechanism rather than by the “[”
utility. At this point, before invoking “[”, the shell already
knows, whether the parameter “arg” is set or unset, so it seems to
be rather pointless to me to transform that knowledge into the
string “set” resp. “” or—in the third case—no string, forget that
knowledge and finally let the “[” utility examine its parameter to
regain it.

Now, if “[” is a non‐built‐in utility, it might be quite costly to
invoke it (using one of the “exec” family of functions).

Therefore, I asked myself, is there a way to make that knowledge
known to the shell script without invoking a non‐built‐in utility?
As you already wrote, there is one:

Invoke the shell built‐in “:” or the probably built‐in “false”,
using the fact, that parameter expansion and empty unquoted word
removal are done before the name of the utility to be invoked is
determined:

“${arg+:} false”

will invoke either “: false” or “false”, depending on whether the
shell variable “arg” is set or unset.

If “false” actually is a non‐built‐in utility, the shell function

“false() { return 1; }”

could be defined to replace it w.r.t. invoking a simple command,
which will avoid calling one of the “exec” family of functions.