[zsh] Inline assignments

[kj]

(NOTE: This question is about ZSH.)

This is yet one more query arising from my seemingly bottomless
confusion over environment/shell variables/parameters.

I've found that in some cases, prefixing a command with one or more
variable assignments seems to have the pleasant effect of temporarily
setting these variables only for the duration of the command's
execution (and available to the command with these temporary values).
In other cases, however, such "inline" assignments are ignored,
and in yet some other cases, they are rejected by zsh as syntax
errors.

Here are some examples of the three possibilities described above.

# current value of PAGER environment variable:
% printenv | grep PAGER
PAGER=/usr/local/bin/less

# inline assignment of PAGER is "seen" by printenv:
% PAGER=/bin/cat printenv | grep PAGER
PAGER=/bin/cat

# ...and by set:
% PAGER=/bin/cat set | grep -a PAGER
PAGER=/bin/cat

# ...and by perl:
% PAGER=/bin/cat perl -e 'print "$ENV{PAGER}\n"'
/bin/cat

# ...but neither by printf nor echo nor print:
% PAGER=/bin/cat printf "$PAGER\n"
/usr/local/bin/less
% PAGER=/bin/cat echo $PAGER
/usr/local/bin/less
% PAGER=/bin/cat print $PAGER
/usr/local/bin/less

# a situation where inline assignments are downright forbidden:
% IFS=$'\n' for i ( $( grep -wl foo * ) ) echo $i
zsh: parse error near `for'

Sometimes the (seemingly) inconsistent behavior surrounding these
inline assignments is downright bewildering. E.g.:

# Inline assignment to shell variable SHELLVAR is seen by set:
% SHELLVAR=42
% set | grep -a SHELLVAR
SHELLVAR=42
% SHELLVAR=24 set | grep -a SHELLVAR
SHELLVAR=24

# ...but inline assignment to shell variable USERNAME isn't:
% set | grep -a USERNAME
USERNAME=kjones
% USERNAME=senojk set | grep -a USERNAME
USERNAME=kjones

# typeset -p does not yield any possible explanation
% typeset -p USERNAME
typeset USERNAME=kjones
% typeset -p SHELLVAR
typeset SHELLVAR=42



I really like these inline assignments, when they work, since often
I want the assignment to apply only for the duration of a command.
(The alternative is to set the variable to the new value before
running the command, and remember to restore the original value
afterwards, but I don't like to risk forgetting to reset the
variable, not to mention the annoyance of having to save the variable
in some temporary variable, whenever there's a non-negligible risk
of my forgetting or mistyping its original value at the time to
restore it).

Unfortunately, as shown above, inline assignments don't always
work. And since I have not figured out why they sometimes work
and sometimes don't, nor can ever remember those individual cases
for which I've determined they work, I end up resorting to subshells,
e.g.

% (PAGER=/bin/cat; echo $PAGER)
/bin/cat

etc.

But using subshells like this has its own pitfalls (typically coming
from forgetting that one is in a subshell) and its own inconveniences.
Therefore, I'd really like to understand inline assignments enough
that I can accurately predict which of them will be seen by the
accompanying command.

Any enlightenment on these questions would be much appreciated.

[Janis Papanagnou]

On 03.03.2013 00:46, kj wrote:
> (NOTE: This question is about ZSH.)

(You'll get equivalent results with ksh.)

>
> This is yet one more query arising from my seemingly bottomless
> confusion over environment/shell variables/parameters.
>
> I've found that in some cases, prefixing a command with one or more
> variable assignments seems to have the pleasant effect of temporarily
> setting these variables only for the duration of the command's
> execution (and available to the command with these temporary values).
> In other cases, however, such "inline" assignments are ignored,
> and in yet some other cases, they are rejected by zsh as syntax
> errors.
>
> Here are some examples of the three possibilities described above.

[ snip a lot of examples ]

>
> Any enlightenment on these questions would be much appreciated.

I suppose reading the man page will answer at least some (maybe all?)
of your questions; look for
* handling of special shell built-ins
* syntactical appropriateness for non-primitive (constructed) commands
* handling of (zsh-specific? read-only) built-in environment variables
* etc.
(If the zsh man page doesn't have the information you may resort to
http://www2.research.att.com/sw/download/man/man1/ksh.html)

Janis

[Barry Margolin]

In article <kgu305$ahu$1...@reader1.panix.com>, kj <no.e...@please.post>
wrote:

> (NOTE: This question is about ZSH.)
>
> This is yet one more query arising from my seemingly bottomless
> confusion over environment/shell variables/parameters.

Yes, it is.

>
> I've found that in some cases, prefixing a command with one or more
> variable assignments seems to have the pleasant effect of temporarily
> setting these variables only for the duration of the command's
> execution (and available to the command with these temporary values).

That's precisely what it's supposed to do. It's part of POSIX shell
syntax (and was in Bourne shell). Prefxing a command with variable
assignments sets those *environment* variables in the child process of
the command.

Note that it does NOT set the shell parameters in the original shell
process, it just effects the child processes.

set is a shell built-in, not an external command. Since the environment
variable is only set in the child process, it's not affected.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Alan Curry]

In article <kgu305$ahu$1...@reader1.panix.com>, kj <no.e...@please.post> wrote:
>

>Here are some examples of the three possibilities described above.
>
> # current value of PAGER environment variable:
> % printenv | grep PAGER
> PAGER=/usr/local/bin/less
>
> # inline assignment of PAGER is "seen" by printenv:
> % PAGER=/bin/cat printenv | grep PAGER
> PAGER=/bin/cat

The assignment modifies the environment of the printenv command; it prints
its environment.

>
> # ...and by set:
> % PAGER=/bin/cat set | grep -a PAGER
> PAGER=/bin/cat

Just like printenv.

>
> # ...and by perl:
> % PAGER=/bin/cat perl -e 'print "$ENV{PAGER}\n"'
> /bin/cat

Mostly like the previous cases.

>
> # ...but neither by printf nor echo nor print:
> % PAGER=/bin/cat printf "$PAGER\n"
> /usr/local/bin/less

This is different. You aren't asking printf to output the value of an
environment variable. You can't ask it to do that. It doesn't know how. It
only prints its arguments.

You're constructing an argument for the printf command by expanding the
current value of $PAGER. Then you're modifying the environment, which has no
effect because the command doesn't use the environment.

If you postpone the evaluation of $PAGER until a later stage...

% PAGER=/bin/cat eval 'printf "$PAGER\n"'
/bin/cat

> % PAGER=/bin/cat echo $PAGER
> /usr/local/bin/less
> % PAGER=/bin/cat print $PAGER
> /usr/local/bin/less

These are the same. Expanding the $PAGER happens before the modification of
the environment. You can see that in all the commands which behave this way,
you have a "$PAGER" being expanded by the shell passed to a command that
doesn't care about the environment, where in the others you have a command
that looks at the environment and/or shell variables itself (including perl,
where you used $ENV{PAGER} and the perl code is quoted with apostrophes so
the shell won't try to expand it).

>
> # a situation where inline assignments are downright forbidden:
> % IFS=$'\n' for i ( $( grep -wl foo * ) ) echo $i
> zsh: parse error near `for'

That syntax only works on simple statements. Maybe a future zsh will make it
work on compound statements too.

But there is a fairly brief way of line-splitting command substitution:

for i in ${(f)"$(cmd)"} ...

>
>Sometimes the (seemingly) inconsistent behavior surrounding these
>inline assignments is downright bewildering. E.g.:
>
> # Inline assignment to shell variable SHELLVAR is seen by set:
> % SHELLVAR=42
> % set | grep -a SHELLVAR
> SHELLVAR=42
> % SHELLVAR=24 set | grep -a SHELLVAR
> SHELLVAR=24
>
> # ...but inline assignment to shell variable USERNAME isn't:
> % set | grep -a USERNAME
> USERNAME=kjones
> % USERNAME=senojk set | grep -a USERNAME
> USERNAME=kjones

USERNAME has its own little section in zshparam(1). assigning to it is
magical: it actually tries to change your process's uid! A shell running as
root can drop privileges by assigning a new value to USERNAME, or spawn a
child process with different privileges with the assignment-before-command
syntax. If you assign an invalid username or you don't have sufficient
privileges, the assignment fails.

Your inability to change USERNAME is unrelated to your user of the
assignment-before-command syntax. Simple assignments behave the same way:

% echo $USERNAME
pacman
% USERNAME=root
zsh: failed to change group ID: operation not permitted
% echo $USERNAME
pacman
% USERNAME=asdfghjk
% echo $USERNAME
pacman

As you can see, the "username doesn't exist" case fails silently, while the
"insufficient privilege" case tells you what went wrong. From that I can
conclude that you didn't actually create an account called senojk...

--
Alan Curry

[kj]

In <barmar-E40B20....@news.eternal-september.org> Barry Margolin <bar...@alum.mit.edu> writes:

>In article <kgu305$ahu$1...@reader1.panix.com>, kj <no.e...@please.post>
>wrote:

>> I've found that in some cases, prefixing a command with one or more
>> variable assignments seems to have the pleasant effect of temporarily
>> setting these variables only for the duration of the command's
>> execution (and available to the command with these temporary values).

>That's precisely what it's supposed to do. It's part of POSIX shell
>syntax (and was in Bourne shell). Prefxing a command with variable
>assignments sets those *environment* variables in the child process of
>the command.

>Note that it does NOT set the shell parameters in the original shell
>process, it just effects the child processes.

What you describe here is *not* what I find puzzling, but rather
the fact that it seems to be true only for *some* child processes
not for others. For example it does not seem to hold for /bin/echo
nor for /usr/bin/printf:

% PAGER=/bin/cat /bin/echo $PAGER
/usr/local/bin/less
% PAGER=/bin/cat /usr/bin/printf "$PAGER\n"
/usr/local/bin/less

What I am trying to ascertain are the rules that determine *which*
child processes are affected by inline assignments and which aren't.

>> Sometimes the (seemingly) inconsistent behavior surrounding these
>> inline assignments is downright bewildering. E.g.:
>>
>> # Inline assignment to shell variable SHELLVAR is seen by set:
>> % SHELLVAR=42
>> % set | grep -a SHELLVAR
>> SHELLVAR=42
>> % SHELLVAR=24 set | grep -a SHELLVAR
>> SHELLVAR=24
>>
>> # ...but inline assignment to shell variable USERNAME isn't:
>> % set | grep -a USERNAME
>> USERNAME=kjones
>> % USERNAME=senojk set | grep -a USERNAME
>> USERNAME=kjones

>set is a shell built-in, not an external command. Since the environment
>variable is only set in the child process, it's not affected.

Yes, but the point of the question was that your statement here
seems to be true *only* for USERNAME (and maybe other parameters),
*not* for SHELLVAR, as shown above, even though both are shell
variables with the same properties (at least as far as typeset -p
is concerned). It is this apparent inconsistency that I'm trying
to sort out.

[kj]

In <kguk2k$qpg$1...@speranza.aioe.org> pac...@kosh.dhis.org (Alan Curry) writes:

>In article <kgu305$ahu$1...@reader1.panix.com>, kj <no.e...@please.post> wrote:

>You're constructing an argument for the printf command by expanding the
>current value of $PAGER. Then you're modifying the environment, which has no
>effect because the command doesn't use the environment.

>If you postpone the evaluation of $PAGER until a later stage...

>% PAGER=/bin/cat eval 'printf "$PAGER\n"'
>/bin/cat

Ah! That explains a lot!

>> % PAGER=/bin/cat echo $PAGER
>> /usr/local/bin/less
>> % PAGER=/bin/cat print $PAGER
>> /usr/local/bin/less

>...You can see that in all the commands which behave this way,

>you have a "$PAGER" being expanded by the shell passed to a command that
>doesn't care about the environment, where in the others you have a command
>that looks at the environment and/or shell variables itself (including perl,
>where you used $ENV{PAGER} and the perl code is quoted with apostrophes so
>the shell won't try to expand it).

OK, this is crystal-clear. Thank you!

As I read your explanation I was reminded of other times when I've
been unpleasantly surprised by the way the shell behaved, and that
ultimately came down to an inaccurate picture of the sequence of
events.

I wish there was a way to show this sequence of events in detail
(something like xtrace, but at a deeper level). Of course, for
the examples above such a trace would probably be overkill, but
there are other far more complicated situations (with aliases,
functions, scripts, and subshells flying around) where I find it
almost impossible to keep track of what happens when... Nor have
I managed to find a description of this sequence of events in the
zsh man page that is sufficiently complete to render such a tool
superfluous. (Which is not to say that the information is not
there, though; after years of failing to find the answers to my
questions in the zsh man page I've come to the conclusiong that it
is organized completely orthogonally to the way my brain proceeds.)

>> # a situation where inline assignments are downright forbidden:
>> % IFS=$'\n' for i ( $( grep -wl foo * ) ) echo $i
>> zsh: parse error near `for'

>That syntax only works on simple statements. Maybe a future zsh will make it
>work on compound statements too.

>But there is a fairly brief way of line-splitting command substitution:

>for i in ${(f)"$(cmd)"} ...

Thanks, that's *great* to know!

>USERNAME has its own little section in zshparam(1). assigning to it is
>magical: it actually tries to change your process's uid! A shell running as
>root can drop privileges by assigning a new value to USERNAME, or spawn a
>child process with different privileges with the assignment-before-command
>syntax. If you assign an invalid username or you don't have sufficient
>privileges, the assignment fails.

>Your inability to change USERNAME is unrelated to your user of the
>assignment-before-command syntax. Simple assignments behave the same way:

>% echo $USERNAME
>pacman
>% USERNAME=root
>zsh: failed to change group ID: operation not permitted
>% echo $USERNAME
>pacman
>% USERNAME=asdfghjk
>% echo $USERNAME
>pacman

>As you can see, the "username doesn't exist" case fails silently, while the
>"insufficient privilege" case tells you what went wrong. From that I can
>conclude that you didn't actually create an account called senojk...

OK, this is very illuminating.

Thanks again!

[Eric]

On 2013-03-03, kj <no.e...@please.post> wrote:
<snip>

> What you describe here is *not* what I find puzzling, but rather
> the fact that it seems to be true only for *some* child processes
> not for others. For example it does not seem to hold for /bin/echo
> nor for /usr/bin/printf:
>
> % PAGER=/bin/cat /bin/echo $PAGER
> /usr/local/bin/less
> % PAGER=/bin/cat /usr/bin/printf "$PAGER\n"
> /usr/local/bin/less
>
> What I am trying to ascertain are the rules that determine *which*
> child processes are affected by inline assignments and which aren't.

There is no such rule. In both those command lines the substitution for
$PAGER is done by the _interactive_ shell before anything on the line is
actually run, including the assignment. So of course you see the value
as it was before the inline assignment.

<snip>

>>set is a shell built-in, not an external command. Since the environment
>>variable is only set in the child process, it's not affected.
>
> Yes, but the point of the question was that your statement here
> seems to be true *only* for USERNAME (and maybe other parameters),
> *not* for SHELLVAR, as shown above, even though both are shell
> variables with the same properties (at least as far as typeset -p
> is concerned). It is this apparent inconsistency that I'm trying
> to sort out.

Firstly, any parameter that is special to a particular shell may behave
differently to non-special parameters.

Secondly, it seems that zsh, unlike other shells, does transmit inline
assignments to builtin commands (or maybe set is just implemented
differently in zsh). Either was it doesn't matter. It applies for any
non-special parameter, and there is never any reason to apply inline
assignments to a builtin anyway.

Sadly, I can't find anything on zsh that explains the behaviour with
set, so we need a zsh _expert_ for that.

Other than that everything is behaving as it is supposed to, and the
only thing you need to know is that substitutions (all types in the
order specified in "man zsh") happen _before_ the command line is
executed.

Eric
--
ms fnd in a lbry

[Janis Papanagnou]

On 03.03.2013 21:28, Eric wrote:
> [...]

>
> Secondly, it seems that zsh, unlike other shells, does transmit inline
> assignments to builtin commands (or maybe set is just implemented
> differently in zsh). Either was it doesn't matter. It applies for any
> non-special parameter, and there is never any reason to apply inline
> assignments to a builtin anyway.

$ zsh -c 'whence -v printf'
printf is a shell builtin

$ zsh -c 'LANG=C printf "%f\n" 3.14'
3.140000

$ zsh -c 'printf "%f\n" 3.14'
3,140000


Janis

> [...]