Race condition with pipe expression in process substitution (ksh, bash, but not zsh)

[Janis Papanagnou]

I observed a race condition in a script with the following structure

p1 | tee >( p3 | p2 >out2 ) >( p2 >out1 )
#sleep 1
p4

p1, p2, p3, p4, are all shell scripts. p4 operates on out1 and out2.

Executing above with ksh and bash:
The observed effect is that out2 will not be available in time for
processing through p4; the file out2 will be empty while p4 is
running, but after above script terminates out2 contains the data.
Adding a 'sleep 1' at the indicated place will produce the correct
result; out2 is then available in time for processing by p4.

Executing above with zsh:
It produces the correct result; file out2 is always available for
processing by p4; the 'sleep 1' is not necessary.

It seems that ksh and bash do no synchronisation of the two lines
of commands, while zsh does. ksh and bash seem to execute the
pipeline 'p3 | p2 >out2' asynchronously, despite the sequencing
operator (the newline) between the two lines of commands.

Now I'd be interested to know whether that has to be considered to
be a bug (in ksh and bash, but not in zsh), or some misconception
that I have about how it should work.

Janis

[Barry Margolin]

In article <kbg2n3$a4e$1...@news.m-online.net>,

We had a discussion of this a little over a year ago:

https://groups.google.com/forum/?fromgroups=#!topic/comp.unix.shell/GqLNz
UA4ulA

zsh waits, bash doesn't. Arguments were given in that thread for both
behaviors, although I'm on your side (if you don't want to wait, use &
to put the processes in the background).

I don't think process substitution is a POSIX feature, so there's no
standard to say which one is right. So it's not clear that either way
could be considered a "bug". Although someone said that zsh introduced
the feature, so I suppose they could be considered the definer of how it
"should" work.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Janis Papanagnou]

Thanks for the link!

>
> zsh waits, bash doesn't. Arguments were given in that thread for both
> behaviors, although I'm on your side (if you don't want to wait, use &
> to put the processes in the background).

David Korn seems to agree as well, and I think he wants to fix that
(in ksh). (I assume originally that had been defined asynchronously
for optimisation reasons.)

The question had been brought up whether the synchronous behaviour
is only really necessary in above >(...) case, and whether it could
stay asynchronous in case of <(...). I thought a bit about that but
could not find a counter example; do you see where <(...) would fail
if still provided in an asynchronous way?

The only concern that I have is that it *might* be better to define
the semantics of both constructs equal, and demand an explicit '&'
if asynchrony is required. But I have no strong opinion about that.

>
> I don't think process substitution is a POSIX feature, so there's no
> standard to say which one is right. So it's not clear that either way
> could be considered a "bug". Although someone said that zsh introduced
> the feature, so I suppose they could be considered the definer of how it
> "should" work.

There seems to be another opinion about the origin. According to
David Korn, it was first suggested by Doug McIlroy in the late '80s
and was first implemented by him in ksh88.

Janis

[Janis Papanagnou]

On 04.01.2013 21:05, Janis Papanagnou wrote:
> [process substitution]

> There seems to be another opinion about the origin. According to
> David Korn, it was first suggested by Doug McIlroy in the late '80s
> and was first implemented by him in ksh88.

Implemented by "him", should actually mean by "David Korn" (not McIlroy).
(Just to clarify.)

> Janis
>

[Dan Douglas]

I've read that the origin was rc, though the syntax was something like <{cmds}. I can't find any resource to cite on that ATM.