Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

set -e in (subshells) does not work independently of outer context

1,018 views
Skip to first unread message

Jim Avera

unread,
Jan 24, 2012, 4:53:14 AM1/24/12
to bug-...@gnu.org, ba...@packages.debian.org
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash'
-DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include
-I../bash/lib -g -O2 -Wall
uname output: Linux lxjima 3.0.0-15-generic #25-Ubuntu SMP Mon Jan 2
17:44:42 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.2
Patch Level: 10
Release Status: release

Description:
set -e in (subshells) should be independent of surrounding context.
The man page says "[set -e] applies to the shell environment and
each subshell environment separately",
but actually set -e is prevented from working in a (subshell) if it is
disabled in the
surrounding context.

Repeat-By:
set +e
(
set -e
cat /non/existent/path
echo "Did not abort. Isn't this a bug?"
) || true

Discussion:
Set -e rightly ignores errors in the outer shell because of
the || expression, but it is not clear why that should have any effect
inside the (subshell) expression.

For example, if you say
set -e
set -o errtrace
trap 'echo -e "Command failed unexpectedly at \c"; caller 0; exit
1' ERR

then you might hope to find out the location of any unexpected command
failures. However, commands which fail in (subshell code) like above
will not cause the trap because they will not abort the subshell.

In summary, shouldn't set -e in (subshell code) operate independently
of the surrounding context?


Eric Blake

unread,
Jan 24, 2012, 11:41:36 AM1/24/12
to james...@yahoo.com, bug-...@gnu.org, ba...@packages.debian.org
On 01/24/2012 02:53 AM, Jim Avera wrote:
> Description:
> set -e in (subshells) should be independent of surrounding context.
> The man page says "[set -e] applies to the shell environment and
> each subshell environment separately",
> but actually set -e is prevented from working in a (subshell) if it is
> disabled in the
> surrounding context.
>
> Repeat-By:
> set +e
> (
> set -e
> cat /non/existent/path
> echo "Did not abort. Isn't this a bug?"
> ) || true

Alas, POSIX requires this behavior.
http://austingroupbugs.net/view.php?id=52


> In summary, shouldn't set -e in (subshell code) operate independently
> of the surrounding context?

No. The POSIX description is clear that surrounding context affects
whether set -e is ignored in a subshell.

2) The -e setting shall be ignored when executing the compound
list following the while, until, if, or elif reserved word,
a pipeline beginning with the ! reserved word, or any
command of an AND-OR list other than the last.

My personal advice - don't ever expect 'set -e' to work; there are just
too many pitfalls and buggy shell implementations with differing
interpretations to ever expect it to be useful, even if there is a
standardized documentation for how it should behave, and even if bash
has (finally) caught up to the standards.

--
Eric Blake ebl...@redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org

signature.asc

Chet Ramey

unread,
Jan 24, 2012, 12:03:38 PM1/24/12
to james...@yahoo.com, bug-...@gnu.org, ba...@packages.debian.org, chet....@case.edu
On 1/24/12 4:53 AM, Jim Avera wrote:

> Bash Version: 4.2
> Patch Level: 10
> Release Status: release
>
> Description:
> set -e in (subshells) should be independent of surrounding context.
> The man page says "[set -e] applies to the shell environment and
> each subshell environment separately",
> but actually set -e is prevented from working in a (subshell) if it is
> disabled in the
> surrounding context.

That sentence is cribbed from the Posix description, and means that if
the -e setting is active in a subshell, it affects whether or not that
subshell exits on command failure, but the outer shell may or may not
exit depending on its context.

The Posix example is

set -e; (false; echo one) | cat; echo two

where the subshell inherits set -e because it's not in a place where it's
ignored, and exits due to the `false' failing, but does not affect the
exit status of the pipeline. The failure of the subshell does not imply
the failure of the pipeline.

As Eric said, the other parts of the Posix description make it clear that
the `ignoring set -e' status is inherited by subshells.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU ch...@case.edu http://cnswww.cns.cwru.edu/~chet/

Eric Blake

unread,
Jan 25, 2012, 11:15:29 AM1/25/12
to james...@yahoo.com, Bash - Bug
[re-adding the list - also, please don't top-post on technical lists]

On 01/24/2012 06:53 PM, Jim Avera wrote:
> Thanks for the quick reply. I read the referenced article and don't
> think it says that at all.
> (http://austingroupbugs.net/view.php?id=52). In fact it seems to imply
> that -e *should* work inside subshell commands even if -e is otherwise
> being ignored (presumably because it is in a conditional context, etc.):
>
> <quote>
> 3) If the exit status of a compound command **other than a subshell**
> command was the result of a failure while -e was being ignored, then -e
> shall not apply to this command. </quote> [emphasis added]
>
> So what is the implication of this statement if the compound -is- a
> subshell command? Why was the exception "other than a subshell command"
> specified?
> Presumably -e should apply regardless of whether it was being ignored
> outside in such cases.
>
> I checked, and bash's current behavior matches ksh. So it is compatible.
>
> But as I read the specification, bash (and ksh) do something different.

Consider:

$ ksh -c 'set -e; { false && :; }; echo $?'
1
$ bash -c 'set -e; { false && :; }; echo $?'
1

This echoes 1. Why? Inside the compound command, we have an AND-OR
list; which means that the left half of the list is in a context where
-e is ignored (the 'false'); but based on the semantics of &&, the right
half is not executed. That means the overall compound command (the '{
false && :; }') executed with a failure - but the _reason_ it failed was
due to a failure of the AND-OR command, which was in turn due to the
failure of the simple command 'false' in a context where -e is ignored.
Therefore, the compound command has a non-zero status, but does not
trigger an exit for -e, even though -e is in effect for the overall
compound command.

Now, repeat the test with a subshell:

$ ksh -c 'set -e; ( false && : ); echo $?'
$ bash -c 'set -e; ( false && : ); echo $?'

Notice - no exit status. Why? Because sub-shells are the only compound
command where a non-zero status, caused for any reason (including
reasons where set -e was ignored), is fatal to the parent script.

That's what point 3 is trying to tell you.

Point 3 is _not_ requiring subshells to override the contexts where set
-e is ignored. That is, once you are in a context where -e is ignored,
there is NOTHING you can do to get -e obeyed again, not even a subshell.

$ bash -c 'set -e; if (set -e; false; echo hi); then :; fi; echo $?'
hi
0

Even though we called set -e twice (both in the parent and in the
subshell), the fact that the subshell exists in a context where -e is
ignored (the condition of an if statement), there is nothing we can do
in the subshell to re-enable -e.
signature.asc

Linda Walsh

unread,
Jan 30, 2012, 4:27:58 PM1/30/12
to bug-bash


Chet Ramey wrote:

> As Eric said, the other parts of the Posix description make it clear that
> the `ignoring set -e' status is inherited by subshells.

----
The original POSIX standard made this clear -- in that
it was only a failure of a 'simple' command that resulted' in
an err-exit'.

Since a subshell is not a 'simple command', it would not qualify
just like ((n=0)) isn't a simple command, so shouldn't trigger an
error exit...(but has in some recent versions)....

Is that fixed in bash now?





Eric Blake

unread,
Jan 30, 2012, 4:54:54 PM1/30/12
to Linda Walsh, bug-bash
On 01/30/2012 02:27 PM, Linda Walsh wrote:
>
>
> Chet Ramey wrote:
>
>> As Eric said, the other parts of the Posix description make it clear that
>> the `ignoring set -e' status is inherited by subshells.
>
> ----
> The original POSIX standard made this clear -- in that
> it was only a failure of a 'simple' command that resulted' in
> an err-exit'.

Have you not read the link I pointed to? The whole point of
http://austingroupbugs.net/view.php?id=52 is a claim that the POSIX 2001
wording for 'set -e' was buggy, because it was inconsistent with
traditional practice, and therefore that particular bug report provides
the approved TC1 wording for how 'set -e' must behave, when using only
the shell features documented by POSIX.

> Since a subshell is not a 'simple command', it would not qualify
> just like ((n=0)) isn't a simple command, so shouldn't trigger an
> error exit...(but has in some recent versions)....

The corrected POSIX wording makes it clear that any command, simple or
otherwise, that does not meet the list of three exclusions, causes the
shell to exit. It's just that the three exclusions are quite broad
(anything on the left side of a pipe && or ||, anything in the
conditional statement of a compound command, and anything except
subshells that has non-zero status due to a failed inner command in a
context where set -e did not abort the inner command). Bash 4.2 took
great pains to comply to the POSIX wording in that particular bug
report, seeing as how the original wording in POSIX 2001 was deemed buggy.

>
> Is that fixed in bash now?

Yes, bash 4.2 strives to obey the POSIX wording as set forth in POSIX
bug 52. The fact that earlier bash versions did not obey the later
POSIX wording, and were therefore incompatible with ksh behavior, is a
historical wart due to the wording bug in POSIX.

That said, ((n=0)) is not described by POSIX. Here, bash might do
better to treat ((n=0)) the same way as ksh, even though in isolation,
it is a command exiting with non-zero status.

$ bash -c 'set -e; ((foo=0)); echo $?.$foo'
$ ksh -c 'set -e; ((foo=0)); echo $?.$foo'
1.0

Or, put another way, if POSIX were to standardize (()), then it would
probably have to amend bug 52 to describe whether (()) forms a fourth
exception to the list of commands where set -e does not cause an exit on
non-zero status. But without a standard to follow, the fact that ksh
and bash differ in behavior is a fact of life, and you shouldn't be
using non-POSIX extensions if you want your script to be portable to
both shells.
signature.asc

Linda Walsh

unread,
Feb 8, 2012, 6:31:54 PM2/8/12
to bug-...@gnu.org
Eric Blake wrote:

On 01/30/2012 02:27 PM, Linda Walsh wrote:

Chet Ramey wrote:


As Eric said, the other parts of the Posix description make it clear that
the `ignoring set -e' status is inherited by subshells.

----
The original POSIX standard made this clear -- in that
it was only a failure of a 'simple' command that resulted' in
an err-exit'.

Have you not read the link I pointed to? The whole point of
[1]http://austingroupbugs.net/view.php?id=52 is a claim that the POSIX 2001
wording for 'set -e' was buggy, because it was inconsistent with
traditional practice, and therefore that particular bug report provides
the approved TC1 wording for how 'set -e' must behave, when using only
the shell features documented by POSIX.

---
A A A The buggy behavior they mentioned had to do with pipes and
subshells.
Then they changed *multiple behaviors* related to "-e", some of which
were
unrelated to the problem they mentioned.
The initial implementation of "-e" was to catch programs that returned
with
non-zero return values so that if you didn't check for them, the
program would
abort.
It was never intended to cause internal shell commands to fail in the
same
way.A The Shell, would know if an error condition had occurred, and
wouldn't
need a global-catch-all switch like "-e", to flag an error.
Example
let a=0
or ((a=0))
The shell would know that an internal math calculation setting a
variable to zero Cannot be an error.A Thus it would never be flaggedA
as an error.
This is the wording of the Austin committee has done -- it has made the
Shell
appear stupid.A -- because the writers of that spec were unthinking
and
didn't restrict their changes to the specific problem at hand.A They
broke
existing practice and broke the existing POSIX SPEC, by creating a new
and incompatible spec, -- that IS NOT POSIX compatible.
Adhering to orders that are wrong, because "it's the 'standard', didn't
work for
Nazi officers, some excuse for not using their brain an realizing theA
'rules' or standard as stated IS wrong.
They even screwed up their example about using "-e" in functions...
They should say not to rely -e in function but instead have shown an
example of
how to use it.
For example - any shell function including my stdlib.shlib,A (which I
won't
claim is bugfree, bug this part did work).
and does
function xxx {
DebugPushA <DebugClass>
....
#before any exit -
DebugPop
}
would have the value of "-e" preserved if that was turned on --
preserving it
into function calls as specified for the specific 'DebugClass'...(as
well as
other Debug options, tracing, (-x), for example).A Not only might they
be preserved, but the flags could be toggled on/off per class. so
-e/-x/-u would pop on for some classes, or off for others, regardless
of globally set flags.
But they stupidly say "just don't use this 'potentially' valuable
development feature, instead well will provide '/dev/null'.A
(Nothing).A A The ethics and morality of those who choose to destroy
and not create replacement, is no greater than that of the arsonist or
architect of destruction - it's simply a matter of 'scale'.
If bash brainlessly complies with brainless orders, then it is fully
responsible for choosing to be brainless.A The same goes for any sheep
who stupidly follow practices without examining details -- like closing
out bugs as 'knee jerk' standard practice' without even examining
recent release notes to note that the
area the bug cropped up in was something that was rewritten with some
incompatible behavior changes.A But that would require responsible and
thoughtful behavior, something that you don't see in the the software
world today.
Everything is about following the path of least resistance and going
for whatever is easiest and cheapest, because it's a race to the
bottom, for quality and features.
Meanwhile the same companies pushing for 0-incremental costs on
software, are those that are pushing for harsher penalties for those
who freely duplicate such software (or content)...A
I see the whole thing as a related and intertwined pathology that is,
in the long run, antithetical to human existence and survival.
More people need to stop being part of the problem and realize that
just because some people say things that don't appear to be pleasant,
doesn't mean they are not true -- they may simply lack the ability to
dissemble as well as others -- a
large disadvantage in today's society.
The Autin's groups pronouncements broke POSIX to the point that I don't
consider their pronouncements to be posix compatible and software that
tries to follow the parts of their docs that reduce functionality *and*
conflict with the original (not due to security faults) are equally
broken.
-l

References

1. http://austingroupbugs.net/view.php?id=52

Chet Ramey

unread,
Feb 8, 2012, 8:42:37 PM2/8/12
to bug-...@gnu.org, Linda Walsh, chet....@case.edu
On 2/8/12 6:31 PM, Linda Walsh wrote:

> Adhering to orders that are wrong, because "it's the 'standard', didn't
> work for
> Nazi officers, some excuse for not using their brain an realizing theA
> 'rules' or standard as stated IS wrong.

I note another instance of proof of Godwin's Law.

Linda Walsh

unread,
Feb 8, 2012, 9:28:11 PM2/8/12
to chet....@case.edu, bug-...@gnu.org
Please note, I didn't compare anyone or their actions to those of Nazi's,

I used them as a historically famous example of someone who claimed to only
be following 'orders', (or the rules, or the standard....whatever!)... and in
a world court, it was deemed that such excuses were not a valid excuse for not
thinking for themselves and realizing that whatever the thing was they'd been
told to do, was 'wrong'. (in their case, more wrong than most), but as I said
before, we are talking a matter principle -- NOT whether someone is behaving in
a Nazi like fashion, THOUGH, if we wanted to compare and look at those who are
engaged in dictatorial actions, and those who are more commonly marginalized as
outsiders, I'm sure such a comparison would be 'pointless' (though ironic).

Chet Ramey

unread,
Feb 8, 2012, 9:42:48 PM2/8/12
to Linda Walsh, bug-...@gnu.org, chet....@case.edu
On 2/8/12 9:28 PM, Linda Walsh wrote:
> Please note, I didn't compare anyone or their actions to those of Nazi's,

This discussion has gone on for a long time; the probability of a
comparison involving Nazis hit 1; ipso facto, Godwin's Law holds.

Linda Walsh

unread,
Feb 8, 2012, 11:15:26 PM2/8/12
to chet....@case.edu, bug-...@gnu.org
Chet Ramey wrote:

> On 2/8/12 9:28 PM, Linda Walsh wrote:
>> Please note, I didn't compare anyone or their actions to those of Nazi's,
>
> This discussion has gone on for a long time; the probability of a
> comparison involving Nazis hit 1; ipso facto, Godwin's Law holds.
----

But that would be similar to saying a similar law about 'the Reds' in the 50's,
or Witches in the 1600's, or devils and demons before that.... it's likely the
last most famous example of a group that had individuals who collectively
evidenced almost any bad behavior we can think of -- so it's only natural, it
would be, the most recent example of 'bad behavior' that would be used that most
people would agree was bad -- they were they losers in the last world war.

They'll probably stay in their position of infamy until we have another 'worse'
enemy (which, let us hope, won't happen, and double hope, won't be the US)...


But can you name any other single group that is so widely known that can be
used as a bad example?

It's hardly noteworthy to say that eventually a conversation, if it goes on long
enough, will make mention of the worst of recent events or the best .. or
anything in between. It would be much more interesting if the conversation
continued until comparisons to devils and demons or original sin, or the evil of
the most jealous god... who purged all gods before him and demands exclusive
veneration. Now those would be interesting comparisons in a conversation, but
Nazi's? That's just recent events... ..

Though ... I suppose, one could think of posix as a replacement? But I doubt
they have the infamy points.




Eric Blake

unread,
Feb 10, 2012, 3:22:31 PM2/10/12
to Linda Walsh, bug-...@gnu.org
On 02/08/2012 04:31 PM, Linda Walsh wrote:
> More people need to stop being part of the problem and realize that
> just because some people say things that don't appear to be pleasant,
> doesn't mean they are not true -- they may simply lack the ability to
> dissemble as well as others -- a
> large disadvantage in today's society.

Rather than whining about bash complying to standards that you think are
less than stellar, you can join the Austin Group and help become part of
the solution of improving those standards. Membership does not cost any
money.

Right now, there is a proposal on the floor to add a new 'set' option to
the shell that would give 'set -e' the ability to be turned back on in
contexts where the historical ksh88 (and thus POSIX 2008 wording)
required it to be silently ignored.

http://austingroupbugs.net/view.php?id=537

| If anything, the way to go about this would be to keep 'set -e'
unchanged in behavior by default, where there are contexts (such as
functions) where reusing 'set -e' has no effect because of the calling
context; and then go about getting consensus among the various shell
implementers about adding a new set option (maybe named 'set -o
errreset'; good luck in finding a short option name that works for all
the shells) that lifts the restriction about contexts where 'set -e' is
ignored. That is:
|
| f() { set -e; false; echo hello; }; if f; then :; fi
|
| would continue to echo hello, but:
|
| f() { set -e; false; echo hello; }; set -o errreset; if f; then :; fi
|
| would let the set -e inside f() take effect, and inhibit the echo. By
doing this, scripts written to the behavior of 0000052 will continue to
work, and new scripts can opt-in to the more intuitive semantics.

I suggest you read up on this, chime in with your thoughts, or even
better, chime in with patches to bash (bash is free software with open
source, after all).

And if you'll read carefully, you might even recognize that I was the
one that wrote that particular comment on the Austin Group bug page. In
other words, I agree that the current behavior of 'set -e' is
unintuitive, but I also feel strongly that the only way to improve
things without breaking existing scripts that have come to rely upon the
documented effects of 'set -e' is to add a new option, where users can
opt-in to more-intuitive behavior. Now, exactly what that behavior
should be, and how to write it up in standardese, is where we need
assistance from you, as the end user, to help develop a working
prototype of the new shell option, and make sure that it meets your
needs, before making it part of the next version of POSIX. It's a lot
easier to standardize something with a reference implementation, after all.
signature.asc
0 new messages