for loop inside heredoc?

[Harry]

Hi All
I have writen a shell script to cycle some servers on
a Linux box using a utility (SCP).

Below is what I have so far.
It works with one parameter, e.g. cycle_server 123.

Is it possible to expand the parameters inside the heredoc
so that it would work for more than one parameters?
If so, how?


# cat cycle_server
scp_num=$1
NODE=`uname -n`
DOMAIN=build
cer_exe=/path/to/exe
SCP=$cer_exe/scpview
USER=my_username
PSWD=my_password
newgrp - d_build << ENDGRP
$SCP <<EOF | /bin/mail -s "server(s) cycled on $DOMAIN on $NODE" m...@myco.ca
$USER
$DOMAIN
$PSWD
select $NODE
cycle -entry $scp_num
#cycle -entry <second_parameter>
#cycle -entry <third_parameter>
# etc
exit
EOF
ENDGRP

TIA

[Janis Papanagnou]

On 03.07.2015 22:10, Harry wrote:
> Hi All
> I have writen a shell script to cycle some servers on
> a Linux box using a utility (SCP).
>
> Below is what I have so far.
> It works with one parameter, e.g. cycle_server 123.
>
> Is it possible to expand the parameters inside the heredoc
> so that it would work for more than one parameters?
> If so, how?

Not directly, but there's a couple possibilities for workarounds.
One is to let the shell do the looping, as in

{
cat <<END
...constant prelude...
END
for par in a b c ...
do
cat <<END
...$par...
END
done
cat <<END
...constant trailor...
END
} | /bin/mail -s ...

Another one is to expand a command or function inside the here-doc,
as in

cat <<END | /bin/mail -s ...
$( create_arguments )
END

(Note: I let the END tags indented for readability; it needs to be
adjusted in real code.)

Janis

[Barry Margolin]

In article <mn6rq6$j42$1...@news.m-online.net>,

Janis Papanagnou <janis_pa...@hotmail.com> wrote:

> On 03.07.2015 22:10, Harry wrote:
> > Hi All
> > I have writen a shell script to cycle some servers on
> > a Linux box using a utility (SCP).
> >
> > Below is what I have so far.
> > It works with one parameter, e.g. cycle_server 123.
> >
> > Is it possible to expand the parameters inside the heredoc
> > so that it would work for more than one parameters?
> > If so, how?
>
> Not directly, but there's a couple possibilities for workarounds.
> One is to let the shell do the looping, as in
>
> {
> cat <<END
> ...constant prelude...
> END
> for par in a b c ...
> do
> cat <<END
> ...$par...
> END

Watch out there. The end token of a heredoc is only recognized when it's
at the left margin. You can't indent it to match the structure of the
rest of the code.

You can use <<-END to allow the end token to be indented with TABS. But
unless you use tabs for your code indentation, it won't line up.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Janis Papanagnou]

On 04.07.2015 00:49, Barry Margolin wrote:
> In article <mn6rq6$j42$1...@news.m-online.net>,
> Janis Papanagnou <janis_pa...@hotmail.com> wrote:
>

[...]

>>
>> {
>> cat <<END
>> ...constant prelude...
>> END
>> for par in a b c ...
>> do
>> cat <<END
>> ...$par...
>> END
>
> Watch out there. The end token of a heredoc is only recognized when it's
> at the left margin. You can't indent it to match the structure of the
> rest of the code.

You obviously seem to missed what I wrote below.

[...]

>>
>> (Note: I let the END tags indented for readability; it needs to be
>> adjusted in real code.)
>>
>> Janis

[...]

[Harry]

I've tried both suggestions from Janis with no avail.
I then tried the following.

# cat -n cycle_server
1 params=($@)
2 NODE=`uname -n`
3 DOMAIN=nuild
4 cer_exe=/path/to/exe
5 SCP=$cer_exe/scpview
6 USER=my_username
7 PSWD=my_password
8 newgrp - d_build << ENDGRP
9 for i in "${params[@]}"
10 do
11 $SCP <<EOF | /bin/mail -s "server cycled on $DOMAIN on $NODE" m...@myco.ca
12 $USER
13 $DOMAIN
14 $PSWD
15 select $NODE
16 server -entry "$i"
17 exit
18 EOF
19 done
20 ENDGRP

Then ran as:
./cycle_server 721
But I got an email with result that implied line #16 were:
server -entry 0
when I expected it would be
server -entry 721
.
Note that line #8 invoked a subshell ...
And I seemed not able to preserve even my first parameter.
Any hint how I could preserve the parameters on the subshell?

Thanks

[Martijn Dekker]

In article <e62dbd13-d140-4760...@googlegroups.com>,

Harry <harryoo...@hotmail.com> wrote:

> Is it possible to expand the parameters inside the heredoc
> so that it would work for more than one parameters?

By default, the $ and ` characters are special in here-documents, so you
can use $variables, but also $(command substitutions). The output of the
command substitution is inserted in the here-document. That means you
can do something like:

command <<EOF
somestuff
$(for var in $things; do
echo "$var"
done)
yetmorestuff
EOF

Hope this helps.

FYI, if you don't want any character to have a special meaning, then
quote (part of) the delimiter word as in: command <<'EOF'

More info:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#
tag_18_07_04

- M.

[Rakesh Sharma]

You would need to escape the $i on line 16 as that variable is being generated
inside the heredoc unlike the others, which are being copied & pasted (by the
parent shell).

Why are you using the "exit" command inside the heredoc?
Also if you happen to pass multiple arguments inside the heredoc, they
will all be coalesced into one long space separated argument. IOW, this approach
is broken due to the bug in the shell. But if you insist on newgrp, all's not
lost. Just interchange the newgrp & for command, so that the newgrp would be
invoked from inside the for loop. Here, you would need the exit command.

Alternatively, the "sg" (=> set group) command is your friend.
Something along these lines....

if [ "$(id -gn)" != "u_build" ]; then
exec sg u_build "$0" ${1+"$@"}
fi

NODE=`uname -n`
DOMAIN=nuild
cer_exe=/path/to/exe
SCP=$cer_exe/scpview
USER=my_username
PSWD=my_password
for i
do
"$SCP" <<EOF | mail -s "server cycled on $DOMAIN on $NODE" m...@myco.ca

$USER
$DOMAIN
$PSWD
select "$NODE"

server -entry "$i"
EOF

-Rakesh

[Harry]

Martijn Dekker wrote...

>By default, the $ and ` characters are special in here-documents, so you
>can use $variables, but also $(command substitutions). The output of the
>command substitution is inserted in the here-document. That means you
>can do something like:
>
>command <<EOF
>somestuff
>$(for var in $things; do
> echo "$var"
>done)
>yetmorestuff
>EOF
>
>Hope this helps.

It works! Thanks a lot.

# store arguments in a special array
args=("$@")
# get number of elements
num_args=${#args[@]}
[...]
newgrp - d_certx << ENDGRP
$SCP <<EOF | /bin/mail -s "server cycled on $DOMAIN on $NODE" m...@myco.ca

$USER
$DOMAIN
$PSWD
select $NODE

$(for (( i=0;i<$num_args;i++)); do
echo server -entry ${args[${i}]}
done
)
exit
EOF
ENDGRP

[Martijn Dekker]

In article <5nZlx.72469$rZ2....@fx20.iad>,

harryoo...@hotmail.co_ (Harry) wrote:

> $(for (( i=0;i<$num_args;i++)); do
> echo server -entry ${args[${i}]}
> done
> )

Yep, that works. It can be made a bit simpler, though.

Since "printf '%s\n'" print each argument on a separate line, repeating
all static text that is in the format argument on each line, you can do
this without any loop:

$(printf 'server -entry %s\n' "${args[@]}")

I don't know if you're using the 'args' array and/or 'num_args" variable
for anything else, but if not, you can simplify it further by just using
"$@" directly, as in:

[...]
newgrp - d_certx << ENDGRP
$SCP <<EOF | /bin/mail -s "server cycled on $DOMAIN on $NODE" m...@myco.ca
$USER
$DOMAIN
$PSWD
select $NODE

$(printf 'server -entry %s\n' "$@")
exit
EOF
ENDGRP

- M.

[Harry]

> $(printf 'server -entry %s\n' "$@")

So this cmd splits itself to 3, when I enter 3 parameters.
I didn't know it could do that.
Amazing!

[Rakesh Sharma]

Make sure to quote the arguments to avoid
word splitting's mal-effects. That is,

[Martijn Dekker]

In article <271492cd-5d29-4db9...@googlegroups.com>,

Rakesh Sharma <shar...@hotmail.com> wrote:

> Make sure to quote the arguments to avoid
> word splitting's mal-effects. That is,
>
> > > $(printf 'server -entry %s\n' "$@")
>
> $(printf 'server -entry "%s"\n' "$@")

That argument was already quoted with single quotes. The double quotes
around the %s aren't quoting an argument, but are part of the
single-quoted argument, so they will become part of each line of output.

- M.

[Martijn Dekker]

In article <1e0d8daa-fcac-4762...@googlegroups.com>,

More generally, "$@" (including the double quotes) is a
specially-treated form that actually represents multiple arguments,
namely as many as there are positional parameters in the current
environment.

- M.

[Kaz Kylheku]

The output, however, resembles a command, which suggests that it will
be subject to evaluation by the shell, at which point it matters.

(Whether simply wrapping the interpolated text with double quotes is
sufficient depends on the text, of course.)

[Kaz Kylheku]

What is so amazing about "if more arguments remain, goto top".

It's an ill-conceived hack that will cause a bug if you don't expect it, and
doesn't have the flexibility to iterate over just some delimited part of the
format string, e.g.

# totally imaginary syntax:
$ printf "fixed: %s; iterated pairs:%{ <%s %s>%}\n" 1 2 3 4 5 6 7
fixed: 1; iterated pairs <2 3> <4 5> <6 7>

Still far short of "amazing".

[Rakesh Sharma]

On Tuesday, 7 July 2015 07:45:52 UTC+5:30, Kaz Kylheku wrote:
> On 2015-07-07, Martijn Dekker <mar...@inlv.demon.nl> wrote:
> > In article <271492cd-5d29-4db9...@googlegroups.com>,
> > Rakesh Sharma <shar...@hotmail.com> wrote:
> >
> >> Make sure to quote the arguments to avoid
> >> word splitting's mal-effects. That is,
> >>
> >> > > $(printf 'server -entry %s\n' "$@")
> >>
> >> $(printf 'server -entry "%s"\n' "$@")
> >
> > That argument was already quoted with single quotes. The double quotes
> > around the %s aren't quoting an argument, but are part of the
> > single-quoted argument, so they will become part of each line of output.
>
> The output, however, resembles a command, which suggests that it will
> be subject to evaluation by the shell, at which point it matters.

The output, howsoever it appears, will be sent to the shell for evaluation
as it's enrobed in $().
And then if the shell can't make sense of it shall barf.

> (Whether simply wrapping the interpolated text with double quotes is
> sufficient depends on the text, of course.)

You're absolutely correct. We are in the eval land here, so it's the user's
responsibility to quote right before handing over to the shell. In such
a scenario, it's best to use single quotes. Why? because no character is
special within single quotes. But whatif there's a single quote in the params? We need to special case this otherwise the shell will trip up.
change each ' ---> '\'' wherever present.

$(printf "server -entry '%s'\n" "$@")

Inside each of the parameters contained in "$@" we need to
quote all the single quotes in the manner prescribed above.

For more info
https://groups.google.com/forum/#!searchin/comp.unix.shell/re-quoting$20Rakesh/comp.unix.shell/3PC4RTzaFKw/GSvrn6-ISEIJ

[Martijn Dekker]

In article <20150706...@kylheku.com>,

Kaz Kylheku <k...@kylheku.com> wrote:

> On 2015-07-07, Martijn Dekker <mar...@inlv.demon.nl> wrote:
> > In article <271492cd-5d29-4db9...@googlegroups.com>,
> > Rakesh Sharma <shar...@hotmail.com> wrote:
> >
> >> Make sure to quote the arguments to avoid
> >> word splitting's mal-effects. That is,
> >>
> >> > > $(printf 'server -entry %s\n' "$@")
> >>
> >> $(printf 'server -entry "%s"\n' "$@")
> >
> > That argument was already quoted with single quotes. The double quotes
> > around the %s aren't quoting an argument, but are part of the
> > single-quoted argument, so they will become part of each line of output.
>
> The output, however, resembles a command, which suggests that it will
> be subject to evaluation by the shell, at which point it matters.

True. I wasn't making that assumption.

> (Whether simply wrapping the interpolated text with double quotes is
> sufficient depends on the text, of course.)

Single quotes are probably a sligthly safer bet, as no characters except
the single quotes themselves are special within them.

But, given untrusted and unvalidated data, it's not possible to do this
100% safely without a proper shell-quoting algorithm.

Unfortunately, though every POSIXly shell necessarily contains such an
algorithm for its own internal use, no current shell exposes it as a
user command. So I wrote my own shellquote function, which I posted here
a while ago.

- M.

[Martijn Dekker]

In article <6ab244d2-e19f-4ed0...@googlegroups.com>,

Rakesh Sharma <shar...@hotmail.com> wrote:

> The output, howsoever it appears, will be sent to the shell for evaluation
> as it's enrobed in $().

Actually, that command substitution was part of a here-document, so in
that case, all it does is insert the output into the here-document.

- M.

[Janis Papanagnou]

The posters wanted to point out that the output of the here-document
looked like a command that will be invoked by a shell or other command,
and it might be a significant difference whether a code like

cat <<EOT
$(printf 'server -entry %s\n' "hello world")
EOT

creates a command like

server -entry "hello world"

or (without quotes)

server -entry hello world


Janis

>
> - M.
>

[Jerry Peters]

Bash's printf has the %q format specifier which claims to "quote the
argument in a way that can be reused as shell input". Not exactly
sure what "as shell input" means exactly.

[Chris F.A. Johnson]

On 2015-07-07, Martijn Dekker wrote:
...

> Single quotes are probably a sligthly safer bet, as no characters except
> the single quotes themselves are special within them.

Single quotes are not special within single quotes. There is no way
to include single quotes within single quotes.

--
Chris F.A. Johnson

[Martijn Dekker]

In article <mnhbgp$vg5$2...@dont-email.me>,

Jerry Peters <je...@example.invalid> wrote:

> Bash's printf has the %q format specifier which claims to "quote the
> argument in a way that can be reused as shell input". Not exactly
> sure what "as shell input" means exactly.

Wow. I had overlooked that. That does indeed provide the functionality
of shellquote(). Why they hid that as an option to printf, I have no
idea.

They mean shell-quoting an arbitrary string so that it can be safely
used as a single-argument string literal in shell code -- for instance,
to prepare a variable containing untrusted/unvalidated data for use with
"eval", so there is no possibility of unwanted code execution.

zsh and AT&T ksh have it too (but not pdksh or mksh, which don't even
have a printf built-in).

Thanks,

- M.

[beings...@gmail.com]

commandxyz -noenv<<EOF
echo "INFO - Inside eof"
t_files=("${p_files[@]}")
#copy array
#echo \${t_files[*]}
#all elements from array
#echo \${#t_files[@]}
#array length
for i in \${t_files[@]} ; do
echo -e \$i;
do other stuff \$i;
done
cat $patch_file
git apply $patch_file
EOF

[beingh...@gmail.com]

Harry wrote:>> High All>> ..

>> Is it possible to expand the parameters inside the heredoc>> so that it would work for more than one parameters?

$ for i in top drawer ;do
> rev <<EOF
> $i
> EOF
> done

pot
reward