Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
some questions on shells, pipes, etc.
7 views
Skip to first unread message
Kenneth Bull
Feb 19, 2012, 7:53:12 PM2/19/12
to
1)
Why does "sleep 1000 | ls " not wait 1000 seconds before running ls?
'ls' runs immediately.
How does ls or the shell know that the program on left will never send
"ls" anything on the pipe?
2)
When making filters that work on both files and programs, do we ever
have to consider \t character? The problem is, reading a file the
filter might get the \t character, but reading a tabbed stream from a
program would look like a bunch of spaces instead.
3)
Is this how the shell works if I call an external program "prog"?
Shell forks itself,
The child calls exec "prog"
The parents calls waitpid(child's pid)
?
4) How does "foreground" and "background" work with the shell? Are
there unix systems calls to implement this?
How does the parent shell go into background and push the child into
foreground in question 3 above? I am guessing what really is happening
is just a reshuffling of stdin/stderr/stdout to implement the
foreground/background function?
5)
When we do "prog 1; prog2" in shell, does the shell fork 2 childs to
start with (and 2nd child is forced to wait somehow) or does the shell
have a table of "to be done commands in future"?
6)
Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
"" ] in sh scrips?
Thank you very much
Why does "sleep 1000 | ls " not wait 1000 seconds before running ls?
'ls' runs immediately.
How does ls or the shell know that the program on left will never send
"ls" anything on the pipe?
2)
When making filters that work on both files and programs, do we ever
have to consider \t character? The problem is, reading a file the
filter might get the \t character, but reading a tabbed stream from a
program would look like a bunch of spaces instead.
3)
Is this how the shell works if I call an external program "prog"?
Shell forks itself,
The child calls exec "prog"
The parents calls waitpid(child's pid)
?
4) How does "foreground" and "background" work with the shell? Are
there unix systems calls to implement this?
How does the parent shell go into background and push the child into
foreground in question 3 above? I am guessing what really is happening
is just a reshuffling of stdin/stderr/stdout to implement the
foreground/background function?
5)
When we do "prog 1; prog2" in shell, does the shell fork 2 childs to
start with (and 2nd child is forced to wait somehow) or does the shell
have a table of "to be done commands in future"?
6)
Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
"" ] in sh scrips?
Thank you very much
Message has been deleted
Barry Margolin
Feb 19, 2012, 8:52:47 PM2/19/12
to
In article
<9619fa71-54d6-48ae...@y10g2000vbn.googlegroups.com>,
concurrently, connecting their stdins and stdouts with pipes.
Since ls doesn't read from stdin, it doesn't have anything to wait for.
> 2)
> When making filters that work on both files and programs, do we ever
> have to consider \t character? The problem is, reading a file the
> filter might get the \t character, but reading a tabbed stream from a
> program would look like a bunch of spaces instead.
I don't see why it would. If a program writes a tab character, that's
what you'll read. Tab characters may be converted to spaces by the
terminal driver when writing to the screen, but that doesn't affect a
pipeline.
> 3)
> Is this how the shell works if I call an external program "prog"?
> Shell forks itself,
> The child calls exec "prog"
> The parents calls waitpid(child's pid)
> ?
That's the basic gist of it.
> 4) How does "foreground" and "background" work with the shell? Are
> there unix systems calls to implement this?
> How does the parent shell go into background and push the child into
> foreground in question 3 above? I am guessing what really is happening
> is just a reshuffling of stdin/stderr/stdout to implement the
> foreground/background function?
When the shell is sitting in waitpid(), the child runs and the shell
sits idle, and is in the background. If you run the program in the
background with "prog &", the shell doesn't call waitpid() immediately,
it just goes back to the code that prompts for another command.
There's more complexity involved in a shell with job control. These
make use of process groups -- when the shell forks to run a job, it uses
setpgrp() to make this a new process group. There's also a terminal
ioctl that specifies which process group is the foreground group, and
only this one is allowed to read from the terminal.
>
> 5)
> When we do "prog 1; prog2" in shell, does the shell fork 2 childs to
> start with (and 2nd child is forced to wait somehow) or does the shell
> have a table of "to be done commands in future"?
It forks them one at a time.
> 6)
> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
> "" ] in sh scrips?
I think you mean if [ "$string" == "" ].
>
> Thank you very much
In case $string starts with "-", which would make it look like one of
the operators.
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
<9619fa71-54d6-48ae...@y10g2000vbn.googlegroups.com>,
Kenneth Bull <kennet...@gmail.com> wrote:
> 1)
> Why does "sleep 1000 | ls " not wait 1000 seconds before running ls?
> 'ls' runs immediately.
> How does ls or the shell know that the program on left will never send
> "ls" anything on the pipe?
The shell doesn't care. It just runs all the programs in the pipeline
> 1)
> Why does "sleep 1000 | ls " not wait 1000 seconds before running ls?
> 'ls' runs immediately.
> How does ls or the shell know that the program on left will never send
> "ls" anything on the pipe?
concurrently, connecting their stdins and stdouts with pipes.
Since ls doesn't read from stdin, it doesn't have anything to wait for.
> 2)
> When making filters that work on both files and programs, do we ever
> have to consider \t character? The problem is, reading a file the
> filter might get the \t character, but reading a tabbed stream from a
> program would look like a bunch of spaces instead.
what you'll read. Tab characters may be converted to spaces by the
terminal driver when writing to the screen, but that doesn't affect a
pipeline.
> 3)
> Is this how the shell works if I call an external program "prog"?
> Shell forks itself,
> The child calls exec "prog"
> The parents calls waitpid(child's pid)
> ?
> 4) How does "foreground" and "background" work with the shell? Are
> there unix systems calls to implement this?
> How does the parent shell go into background and push the child into
> foreground in question 3 above? I am guessing what really is happening
> is just a reshuffling of stdin/stderr/stdout to implement the
> foreground/background function?
sits idle, and is in the background. If you run the program in the
background with "prog &", the shell doesn't call waitpid() immediately,
it just goes back to the code that prompts for another command.
There's more complexity involved in a shell with job control. These
make use of process groups -- when the shell forks to run a job, it uses
setpgrp() to make this a new process group. There's also a terminal
ioctl that specifies which process group is the foreground group, and
only this one is allowed to read from the terminal.
>
> 5)
> When we do "prog 1; prog2" in shell, does the shell fork 2 childs to
> start with (and 2nd child is forced to wait somehow) or does the shell
> have a table of "to be done commands in future"?
> 6)
> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
> "" ] in sh scrips?
>
> Thank you very much
In case $string starts with "-", which would make it look like one of
the operators.
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
Eric Sosman
Feb 19, 2012, 8:54:05 PM2/19/12
to
On 2/19/2012 7:53 PM, Kenneth Bull wrote:
> 1)
> Why does "sleep 1000 | ls " not wait 1000 seconds before running ls?
> 'ls' runs immediately.
> How does ls or the shell know that the program on left will never send
> "ls" anything on the pipe?
"ls" never reads its stdin, hence never needs to wait for
> 1)
> Why does "sleep 1000 | ls " not wait 1000 seconds before running ls?
> 'ls' runs immediately.
> How does ls or the shell know that the program on left will never send
> "ls" anything on the pipe?
the pipe to utter anything (not even to utter EOF). Perhaps
you want "sleep 1000 ; ls" instead.
> 2)
> When making filters that work on both files and programs, do we ever
> have to consider \t character? The problem is, reading a file the
> filter might get the \t character, but reading a tabbed stream from a
> program would look like a bunch of spaces instead.
on't. The visual appearance of a tab on a terminal may be one or
more spaces (or simply horizontal space with no defined character
count), but '\t' is still '\t'. Consider: '\n' is still '\n', even
though it "prints" an infinite number of blank spaces at the end of
its line. If you don't worry about the infinite number of spaces
after '\n', you shouldn't worry about what '\t' looks like.
> 3)
> Is this how the shell works if I call an external program "prog"?
> Shell forks itself,
> The child calls exec "prog"
> The parents calls waitpid(child's pid)
> ?
> 4) How does "foreground" and "background" work with the shell? Are
> there unix systems calls to implement this?
the parent shell does or does not wait for the child.
> How does the parent shell go into background and push the child into
> foreground in question 3 above? I am guessing what really is happening
> is just a reshuffling of stdin/stderr/stdout to implement the
> foreground/background function?
what you're really asking about.
> 5)
> When we do "prog 1; prog2" in shell, does the shell fork 2 childs to
> start with (and 2nd child is forced to wait somehow) or does the shell
> have a table of "to be done commands in future"?
don't remember. Scarcely matters, unless you're implementing a
shell -- in which case you get to make your own rules.
> 6)
> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
> "" ] in sh scrips?
> Thank you very much
You're welcome 1) 2) 4) 6).
--
Eric Sosman
eso...@ieee-dot-org.invalid
Nicolas George
Feb 20, 2012, 2:38:43 AM2/20/12
to
Kenneth Bull , dans le message
<9619fa71-54d6-48ae...@y10g2000vbn.googlegroups.com>, a
écrit :
[ $string == "" ] is not correct because == is not the standard operator,
the correct syntax is with a single =.
[ $string = "" ] is not correct, because if $string is empty, then the
command is expanded into [ = "" ], and that is not a valid syntax.
Furthermore, if $string has special characters, all hell may break loose.
[ "$string" = "" ] is correct on a standard shell.
Unfortunately, not all shells at large respect the standard, and some have
been known to be fooled if it happens that $string starts with a dash: they
think we are testing some kind of property on the file named =, and then
there is an empty string they do not understand. Adding an x in front of the
string hides the dash (it is no longer the first char) and solves the
problem.
<9619fa71-54d6-48ae...@y10g2000vbn.googlegroups.com>, a
écrit :
> 6)
> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
> "" ] in sh scrips?
People have answered all questions satisfactorily except this one.
> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
> "" ] in sh scrips?
[ $string == "" ] is not correct because == is not the standard operator,
the correct syntax is with a single =.
[ $string = "" ] is not correct, because if $string is empty, then the
command is expanded into [ = "" ], and that is not a valid syntax.
Furthermore, if $string has special characters, all hell may break loose.
[ "$string" = "" ] is correct on a standard shell.
Unfortunately, not all shells at large respect the standard, and some have
been known to be fooled if it happens that $string starts with a dash: they
think we are testing some kind of property on the file named =, and then
there is an empty string they do not understand. Adding an x in front of the
string hides the dash (it is no longer the first char) and solves the
problem.
Bjarni Juliusson
Feb 20, 2012, 8:35:49 PM2/20/12
to
On 02/20/2012 01:53 AM, Kenneth Bull wrote:
> 1)
> Why does "sleep 1000 | ls " not wait 1000 seconds before running ls?
> 'ls' runs immediately.
If the question is "Why doesn't the shell wait until the first process
> 1)
> Why does "sleep 1000 | ls " not wait 1000 seconds before running ls?
> 'ls' runs immediately.
terminates before starting the other?", one answer is that it would have
to buffer all the output from the first process. The output is
potentially infinite. Starting both processes at once also allows the
user to see the output in real time, without having to wait for all of
it to be ready.
Would there be any benefit from waiting?
> How does ls or the shell know that the program on left will never send
> "ls" anything on the pipe?
> 2)
> When making filters that work on both files and programs, do we ever
> have to consider \t character? The problem is, reading a file the
> filter might get the \t character, but reading a tabbed stream from a
> program would look like a bunch of spaces instead.
prints tabbed text to its stdout is likely to convert the tabs to spaces
before it prints them, and so if another program is reading the stdout
it will see those spaces instead of tab characters?
> 3)
> Is this how the shell works if I call an external program "prog"?
> Shell forks itself,
> The child calls exec "prog"
> The parents calls waitpid(child's pid)
> ?
control, but yes.
> 4) How does "foreground" and "background" work with the shell? Are
> there unix systems calls to implement this?
job control. I can recommend this article, which explains how the
terminal subsystem works:
http://www.linusakesson.net/programming/tty/index.php
> How does the parent shell go into background and push the child into
> foreground in question 3 above? I am guessing what really is happening
> is just a reshuffling of stdin/stderr/stdout to implement the
> foreground/background function?
the TTY device that that process group is now the foreground group. This
is described in detail in the article above.
Bjarni
--
INFORMATION WANTS TO BE FREE
Stephane Chazelas
Feb 21, 2012, 3:19:34 PM2/21/12
to
2012-02-21 19:42:08 +0000, Scott Lurndal:
>
> For a POSIX shell (bash, ksh, et. al.)
>
> [[ "$string" == "" ]]
>
> is preferred due to expressiveness and lower overhead.
[...]
No, that syntax is not POSIX. It was introduced by AT&T ksh88
and a similar syntax (but with variations) can be found in other
implementations of ksh and bash and zsh, but is not specfied by
POSIX.
A misfeature of that syntax is that "==" is actually a pattern
matching operator unless the right hand side is quoted, which is
yet another source of bugs and confusion.
Note that
[ "" = "$string" ] is also portable.
--
Stephane
> Nicolas George <nicolas$geo...@salle-s.org> writes:
> >Kenneth Bull , dans le message
> ><9619fa71-54d6-48ae...@y10g2000vbn.googlegroups.com>, a
> > écrit :
> >> 6)
> >> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
> >> "" ] in sh scrips?
> >
> >People have answered all questions satisfactorily except this one.
> >
> >[ $string == "" ] is not correct because == is not the standard operator,
> >the correct syntax is with a single =.
> >
> >[ $string = "" ] is not correct, because if $string is empty, then the
> >command is expanded into [ = "" ], and that is not a valid syntax.
> >Furthermore, if $string has special characters, all hell may break loose.
> >
> >[ "$string" = "" ] is correct on a standard shell.
>
> For which "standard" == "bourne".
> >Kenneth Bull , dans le message
> ><9619fa71-54d6-48ae...@y10g2000vbn.googlegroups.com>, a
> > écrit :
> >> 6)
> >> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
> >> "" ] in sh scrips?
> >
> >People have answered all questions satisfactorily except this one.
> >
> >[ $string == "" ] is not correct because == is not the standard operator,
> >the correct syntax is with a single =.
> >
> >[ $string = "" ] is not correct, because if $string is empty, then the
> >command is expanded into [ = "" ], and that is not a valid syntax.
> >Furthermore, if $string has special characters, all hell may break loose.
> >
> >[ "$string" = "" ] is correct on a standard shell.
>
>
> For a POSIX shell (bash, ksh, et. al.)
>
> [[ "$string" == "" ]]
>
> is preferred due to expressiveness and lower overhead.
[...]
No, that syntax is not POSIX. It was introduced by AT&T ksh88
and a similar syntax (but with variations) can be found in other
implementations of ksh and bash and zsh, but is not specfied by
POSIX.
A misfeature of that syntax is that "==" is actually a pattern
matching operator unless the right hand side is quoted, which is
yet another source of bugs and confusion.
Note that
[ "" = "$string" ] is also portable.
--
Stephane
Scott Lurndal
Feb 23, 2012, 10:36:48 AM2/23/12
to
not:
The following words may be recognized as reserved words on some implementations (when none of the characters are quoted), causing unspecified results:
[[
]]
function
select
If I recall correctly, there was recently some discussion of this on the austin
mailing list.
scott
Geoff Clare
Feb 24, 2012, 8:29:41 AM2/24/12
to
Scott Lurndal wrote:
> Stephane Chazelas <stephane...@yahoo.fr> writes:
>>2012-02-21 19:42:08 +0000, Scott Lurndal:
> Stephane Chazelas <stephane...@yahoo.fr> writes:
>>2012-02-21 19:42:08 +0000, Scott Lurndal:
>>> For a POSIX shell (bash, ksh, et. al.)
>>>
>>> [[ "$string" == "" ]]
>>>
>>> is preferred due to expressiveness and lower overhead.
>>>
>>> [[ "$string" == "" ]]
>>>
>>> is preferred due to expressiveness and lower overhead.
>>No, that syntax is not POSIX. It was introduced by AT&T ksh88
>>and a similar syntax (but with variations) can be found in other
>>implementations of ksh and bash and zsh, but is not specfied by
>>POSIX.
>>and a similar syntax (but with variations) can be found in other
>>implementations of ksh and bash and zsh, but is not specfied by
>>POSIX.
> Hmmm. I could have sworn that we had added that to POSIX, but I guess
> not:
>
> The following words may be recognized as reserved words on some implementations (when none of the characters are quoted), causing unspecified results:
>
> [[
> ]]
> function
> select
>
> If I recall correctly, there was recently some discussion of this on the austin
> mailing list.
It's part of the white paper being developed as the proposed resolution
> not:
>
> The following words may be recognized as reserved words on some implementations (when none of the characters are quoted), causing unspecified results:
>
> [[
> ]]
> function
> select
>
> If I recall correctly, there was recently some discussion of this on the austin
> mailing list.
for bug 375. For details see
http://austingroupbugs.net/view.php?id=375
scroll down to the attached files and download either the pdf or odt of
Extendingshellconditionals-2011-11-28
--
Geoff Clare <net...@gclare.org.uk>
Scott Lurndal
Feb 21, 2012, 2:42:08 PM2/21/12
to
Nicolas George <nicolas$geo...@salle-s.org> writes:
>Kenneth Bull , dans le message
><9619fa71-54d6-48ae...@y10g2000vbn.googlegroups.com>, a
> écrit :
>> 6)
>> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
>> "" ] in sh scrips?
>
>People have answered all questions satisfactorily except this one.
>
>[ $string == "" ] is not correct because == is not the standard operator,
>the correct syntax is with a single =.
>
>[ $string = "" ] is not correct, because if $string is empty, then the
>command is expanded into [ = "" ], and that is not a valid syntax.
>Furthermore, if $string has special characters, all hell may break loose.
>
>[ "$string" = "" ] is correct on a standard shell.
>Kenneth Bull , dans le message
><9619fa71-54d6-48ae...@y10g2000vbn.googlegroups.com>, a
> écrit :
>> 6)
>> Why do people do if [ "x"$string == "x" ] instead of if[ $string ==
>> "" ] in sh scrips?
>
>People have answered all questions satisfactorily except this one.
>
>[ $string == "" ] is not correct because == is not the standard operator,
>the correct syntax is with a single =.
>
>[ $string = "" ] is not correct, because if $string is empty, then the
>command is expanded into [ = "" ], and that is not a valid syntax.
>Furthermore, if $string has special characters, all hell may break loose.
>
>[ "$string" = "" ] is correct on a standard shell.
For which "standard" == "bourne".
For a POSIX shell (bash, ksh, et. al.)
[[ "$string" == "" ]]
is preferred due to expressiveness and lower overhead.
scott
[[ "$string" == "" ]]
is preferred due to expressiveness and lower overhead.
mark.b...@thales-is.com
May 9, 2012, 8:05:56 AM5/9/12
to
On Tuesday, February 21, 2012 8:19:34 PM UTC, Stephane Chazelas wrote:
> 2012-02-21 19:42:08 +0000, Scott Lurndal:
> >
> 2012-02-21 19:42:08 +0000, Scott Lurndal:
> >
> > For a POSIX shell (bash, ksh, et. al.)
> >
> > [[ "$string" == "" ]]
> >
> > is preferred due to expressiveness and lower overhead.
> [...]
>
> No, that syntax is not POSIX. ...
> >
> > [[ "$string" == "" ]]
> >
> > is preferred due to expressiveness and lower overhead.
> [...]
>
> A misfeature of that syntax is that "==" is actually a pattern
> matching operator unless the right hand side is quoted, which is
> yet another source of bugs and confusion.
Could you clarify that please?
> matching operator unless the right hand side is quoted, which is
> yet another source of bugs and confusion.
i.e. what do you mean by a pattern-matching operator, and so what if it is one?
Also, what do you mean by quoting the RHS and why does this cause problems?
[Excuse my reply to very old thread; I'm doing some catching up]
0 new messages