errexit inconsistent behaviour with pipelines
Marcin Owsiany
I have these three very simple scripts. The only difference between them is the penultimate line of each of them:
| $ head -n 6 just_pipe just_for_loop pipe_and_for_loop
| ==> just_pipe <==
| set -e
| trap 'echo aborted in line $LINENO' ERR
| echo start
| true | false
| echo ended successfully
|
| ==> just_for_loop <==
| set -e
| trap 'echo aborted in line $LINENO' ERR
| echo start
| for a in 1; do false; done
| echo ended successfully
|
| ==> pipe_and_for_loop <==
| set -e
| trap 'echo aborted in line $LINENO' ERR
| echo start
| true | for a in 1; do false; done
| echo ended successfully
I expected that if a simple pipeline failure causes the shell to exit, and a simple for loop failure causes it to exit, then a failure in their combination would also cause an exit. However it is not so.
| $ for script in just_pipe just_for_loop pipe_and_for_loop ;do echo -n "script $script: ";bash $script;done
| script just_pipe: start
| aborted in line 4
| script just_for_loop: start
| aborted in line 4
| script pipe_and_for_loop: start
| ended successfully
| $
This is using bash 3.2-0ubuntu11.
I found a workaround for a similar (same?) problem in the archive:
http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/5f84f1f50b4715ae/0d8283b2c2025dc0?lnk=gst&q=errexit#0d8283b2c2025dc0
However isn't this inconsistency a bug? I also tried this in dash and zsh, and they both exit consistently.
regards,
Marcin
--
Marcin Owsiany <mar...@owsiany.pl> http://marcin.owsiany.pl/
GnuPG: 1024D/60F41216 FE67 DA2D 0ACA FC5E 3F75 D6F6 3A0D 8AA0 60F4 1216
"Every program in development at MIT expands until it can read mail."
-- Unknown
Chet Ramey
> I expected that if a simple pipeline failure causes the shell to exit, and a simple for loop failure causes it to exit, then a failure in their combination would also cause an exit. However it is not so.
You've found a probable bug, but it's not where you think. Each element
of a pipeline is run in a separate execution environment and a separate
process. Even if the child process inherits the errexit flag, the fact
that false returns failure should not affect the parent shell: a pipeline
is not a simple command. You can see this by adding another process to
the pipeline:
set -e
true | false | echo final
echo after
In all cases, `final' and `after' are both displayed.
This bug only occurs when errexit is enabled and the final element of a
pipeline is a simple command that returns a non-zero exit status. I will
devise a fix for the next bash release.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
Chet Ramey, ITS, CWRU ch...@case.edu http://cnswww.cns.cwru.edu/~chet/
Marcin Owsiany
> Marcin Owsiany wrote:
>
> > I expected that if a simple pipeline failure causes the shell to exit, and a simple for loop failure causes it to exit, then a failure in their combination would also cause an exit. However it is not so.
>
> You've found a probable bug, but it's not where you think. Each element
> of a pipeline is run in a separate execution environment and a separate
> process. Even if the child process inherits the errexit flag, the fact
> that false returns failure should not affect the parent shell: a pipeline
> is not a simple command. You can see this by adding another process to
> the pipeline:
>
> set -e
> true | false | echo final
> echo after
>
> In all cases, `final' and `after' are both displayed.
Yes, I am aware that only the exit status of the last process in the
pipeline is ever propagated to the parent shell (unless you set
pipefail, in which case your example above will only print "final" and
not "after"). By the way - is pipefail a new thing? I only noticed this
last week, and I'm using bash for almost a decade :-)
> This bug only occurs when errexit is enabled and the final element of a
> pipeline is a simple command that returns a non-zero exit status.
Well, if the final element in the pipeline is a simple command, all
works as is supposed to. When it's a-not-so-simple command, like a for
or while loop, then it does not terminate the parent.
> I will devise a fix for the next bash release.
Great, thanks!
Paul Jarc
> On Mon, Oct 06, 2008 at 09:22:30AM -0400, Chet Ramey wrote:
>> This bug only occurs when errexit is enabled and the final element of a
>> pipeline is a simple command that returns a non-zero exit status.
>
> Well, if the final element in the pipeline is a simple command, all
> works as is supposed to.
No. It works the way you want it to, but not the way SUS says it
should.
> When it's a-not-so-simple command, like a for or while loop, then it
> does not terminate the parent.
That's the case that's behaving correctly, even though it's not the
way you want it to behave. Neither case should terminate the shell.
Only a simple command should terminate the shell, and a pipeline is
not a simple command, even if its components are. Chet is saying that
the next bash release will fix this bug - meaning that "true | false"
will no longer cause the shell to exit.
paul
Chet Ramey
> way you want it to behave. Neither case should terminate the shell.
> Only a simple command should terminate the shell, and a pipeline is
> not a simple command, even if its components are. Chet is saying that
> the next bash release will fix this bug - meaning that "true | false"
> will no longer cause the shell to exit.
Yes. I'm sorry that I was not as clear as I should have been.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
Chet Ramey, ITS, CWRU ch...@case.edu http://tiswww.tis.case.edu/~chet/
Marcin Owsiany
> > That's the case that's behaving correctly, even though it's not the
> > way you want it to behave. Neither case should terminate the shell.
> > Only a simple command should terminate the shell, and a pipeline is
> > not a simple command, even if its components are. Chet is saying that
> > the next bash release will fix this bug - meaning that "true | false"
> > will no longer cause the shell to exit.
>
> Yes. I'm sorry that I was not as clear as I should have been.
Oh, I certainly didn't expect it to be fixed this way :-)
Will setting pipefail have any effect on this? If not, then is there a
way to make bash abort when one of the pipeline components fails,
without too much additional code?
On the other hand, to match the documentation, you would also have to
fix cases such as:
set -e
true && false
echo done
As they are not simple commands either, right? The manpage even
explicitly says that it "does not exit if the command that fails is
[...] part of a && or ││ list".
Chet Ramey
> On Mon, Oct 06, 2008 at 04:09:49PM -0400, Chet Ramey wrote:
>>> That's the case that's behaving correctly, even though it's not the
>>> way you want it to behave. Neither case should terminate the shell.
>>> Only a simple command should terminate the shell, and a pipeline is
>>> not a simple command, even if its components are. Chet is saying that
>>> the next bash release will fix this bug - meaning that "true | false"
>>> will no longer cause the shell to exit.
>> Yes. I'm sorry that I was not as clear as I should have been.
>
> Oh, I certainly didn't expect it to be fixed this way :-)
>
> Will setting pipefail have any effect on this? If not, then is there a
> way to make bash abort when one of the pipeline components fails,
> without too much additional code?
>
> On the other hand, to match the documentation, you would also have to
> fix cases such as:
>
> set -e
> true && false
> echo done
>
> As they are not simple commands either, right? The manpage even
> explicitly says that it "does not exit if the command that fails is
> [...] part of a && or ││ list".
That's true, except every other shell behaves this way. It works for
compatibility. I'm going to have to audit this stuff, at the very
least.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
Chet Ramey, ITS, CWRU ch...@case.edu http://cnswww.cns.cwru.edu/~chet/
Chet Ramey
>> Will setting pipefail have any effect on this? If not, then is there a
>> way to make bash abort when one of the pipeline components fails,
>> without too much additional code?
Not easily.
>> On the other hand, to match the documentation, you would also have to
>> fix cases such as:
>>
>> set -e
>> true && false
>> echo done
>>
>> As they are not simple commands either, right? The manpage even
>> explicitly says that it "does not exit if the command that fails is
>> [...] part of a && or ││ list".
>
> That's true, except every other shell behaves this way. It works for
> compatibility. I'm going to have to audit this stuff, at the very
> least.
I should add that it's pretty much unanimous consensus among the `major'
shell implementers that this ambiguous language is a defect in the
standard, and the failure of the rightmost command in an AND or OR list
should cause the shell to exit.
I think the current behavior, once the pipeline stuff is fixed, is
pretty much correct.