[PATCH] fix $(..) and $((..)) syntax highlighting for /bin/sh
Pádraig Brady
are marked as errors. These constructs are POSIX compliant
and supported by dash and bash at least.
I contend that erroneously warning users away from these valid
constructs is much worse than not flagging them as errors
on the dwindling non POSIX compliant shells.
thanks,
Pádraig.
Charles E. Campbell, Jr.
Read :help ft-sh-syntax , in particular lines 2419-2435 . FYI --
bash is not "dwindling".
Regards,
Chip Campbell
Pádraig Brady
You misunderstand I think. Obviously bash is not dwindling :)
$() is OK for bash, dash, ash, ...
It's only not OK for old Borne shells.
Therefore I think we should not flag this POSIX syntax
as an error by default for all /bin/sh scripts.
thanks,
Pádraig.
James Vega
>
> Charles E. Campbell, Jr. wrote:
> > Read :help ft-sh-syntax , in particular lines 2419-2435 . FYI --
> > bash is not "dwindling".
>
> You misunderstand I think. Obviously bash is not dwindling :)
> $() is OK for bash, dash, ash, ...
> It's only not OK for old Borne shells.
And highlighting is done for Borne shell by default. If the script is
named or has a shebang line to indicate it is something other than
Borne, that alternate shell syntax is used. You can also specify the
default shell syntax in your ~/.vimrc.
This is intentional behavior to encourage the user to define their
default shell syntax and is brought on by the many scripts that use
"#!/bin/sh" when they're not actually writing Borne shell syntax. This
is all nicely explained in the help topic that Chip pointed out.
--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jame...@jamessan.com>
Tony Mechelynck
#!/bin/sh scripts are supposed to be for sh (the "old Bourne shell"),
not bash or ksh. If you want them to be highlighted as bash scripts,
then replace #!/bin/sh by #!/bin/bash. Quoting lines 2431 to 2435 of
syntax.txt:
> If there's no "#! ..." line, and the user hasn't availed himself/herself of a
> default sh.vim syntax setting as just shown, then syntax/sh.vim will assume
> the Bourne shell syntax. No need to quote RFCs or market penetration
> statistics in error reports, please -- just select the default version of the
> sh your system uses in your <.vimrc>.
Best regards,
Tony.
--
We don't know who discovered water, but we're certain it wasn't a
fish.
Pádraig Brady
> #!/bin/sh scripts are supposed to be for sh (the "old Bourne shell"),
> not bash or ksh. If you want them to be highlighted as bash scripts,
> then replace #!/bin/sh by #!/bin/bash. Quoting lines 2431 to 2435 of
> syntax.txt:
>
>> If there's no "#! ..." line, and the user hasn't availed himself/herself of a
>> default sh.vim syntax setting as just shown, then syntax/sh.vim will assume
>> the Bourne shell syntax. No need to quote RFCs or market penetration
>> statistics in error reports, please -- just select the default version of the
>> sh your system uses in your <.vimrc>.
It seems like others have reported this issue then.
I would suggest that now, /bin/sh usually means POSIX compliant sh as per:
http://www.opengroup.org/onlinepubs/009695399/
For example if I want to write a script that runs on fedora ( /bin/sh -> bash )
or ubuntu ( /bin/sh -> dash ), then it needs to be a #!/bin/sh script.
Perhaps you just need to change the default now so that is_posix=1
and if one really wants to write old bourne shell script they can
set is_posix=0 in their .vimrc
Note I think this is important as by default, the very useful
$(..) and $((..)) constructs are highlighted as problematic and
people can very easily think they're bash specific etc.
cheers,
Pádraig.
Pádraig Brady
> Perhaps you just need to change the default now so that is_posix=1
> and if one really wants to write old bourne shell script they can
> set is_posix=0 in their .vimrc
>
> Note I think this is important as by default, the very useful
> $(..) and $((..)) constructs are highlighted as problematic and
> people can very easily think they're bash specific etc.
I had a quick look around and noticed that this was reported previously:
http://url.ie/g7d
https://bugs.launchpad.net/ubuntu/+source/vim/+bug/116893
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=361177
So the question boils down to: should we change is_posix=1 by default?
Chip states in one of the threads above that he doesn't want
to change that for backwards compatibility reasons.
I can see the reason for that, but the downside is that
on the vast minority of systems certain POSIX constructs
will not be flagged as errors in vim. The upside though
is that for the vast majority of systems, valid POSIX
constructs will not erroneously be flagged as errors.
cheers,
Pádraig.
p.s. shouldn't the flag have been called is_posix_sh ?
Tony Mechelynck
It might have been called sh_is_posix, but it's too late for that --
upwards compatibility again.
As for "erroneously" -- I agree with Bram that #!sh is supposed to mean
the original Bourne shell, not Bourne-again, not Korn, and quite
possibly before POSIX was invented. Anything which the original Bourne
shell did not support should not be used in a script starting #!sh (or
#!/bin/sh etc.) or else someday you'll feed one of these scripts to a
surviving copy of the original Bourne (not-again) shell and be surprised
that it doesn't work.
Best regards,
Tony.
--
Don't let your mind wander -- it's too little to be let out alone.
Ag. D. Hatzimanikas
While compatibility is an important factor, at the same time could be a
block to evolution.
The fact is that 'sh' *is* the standard interpreter [1] and since $((..))
construct is a valid POSIX syntax shouldn't be treated as an error by
default.
I have the "g:is_posix = 1" setting in one of the initialized files
because my 'sh' is a symbolic link to 'dash' which is a standard POSIX
shell [2].
And I choose 'dash' because I wanted to have valid POSIX scripts that can
be work in every system that has a POSIX shell; and is generally
accepted that 'sh' should be symbolic link to a POSIX compliant shell. If
it isn't (a POSIX compliant shell), then it's a system error, or this
system doesn't care about standards and the $((...) should be treated as
an error in this system and it's the time for "g:is_posix = 0".
So I have to agree with Pádraig if my opinion matters.
1. http://www.opengroup.org/onlinepubs/009695399/utilities/sh.html
2. http://en.wikipedia.org/wiki/Debian_Almquist_shell
> Best regards,
> Tony.
Regards,
Ag.
Pádraig Brady
> As for "erroneously" -- I agree with Bram that #!sh is supposed to mean
> the original Bourne shell, not Bourne-again, not Korn, and quite
> possibly before POSIX was invented. Anything which the original Bourne
> shell did not support should not be used in a script starting #!sh (or
> #!/bin/sh etc.) or else someday you'll feed one of these scripts to a
> surviving copy of the original Bourne (not-again) shell and be surprised
> that it doesn't work.
While forwards compatibility is nice it's not usually the case.
Are you suggesting that #!/usr/bin/python #!/usr/bin/perl etc. scripts
should only be written to the first released version of those languages?
The fact is that #!/bin/sh scripts are run by a POSIX sh interpreter
on the vast majority of systems. That dictates what the default should be IMHO.
cheers,
Pádraig.
p.s. why is "print" highlighted as a keyword in sh scripts. grrr.
James Vega
> The fact is that #!/bin/sh scripts are run by a POSIX sh interpreter
> on the vast majority of systems. That dictates what the default should be IMHO.
You're assuming recent systems. There are still plenty of systems out
there where this isn't necessarily the case.
As stated previously, it's common practice for people to use "#!/bin/sh"
regardless of the shell actually providing that file. What /bin/sh is
on your system is probably different than on my system which is probably
different than what is on someone's AIX system, etc. Individual
distributions of OSes may let a default since they have control over the
common configuration.
I have to agree with Chip here that the upstream version should default
to Bourne as that is most likely to cause people to actually configure
the highlighting to work properly for the variant of /bin/sh that *they*
are using.
Rhialto
> I have to agree with Chip here that the upstream version should default
> to Bourne as that is most likely to cause people to actually configure
> the highlighting to work properly for the variant of /bin/sh that *they*
> are using.
But the current Bourne shell *is* a POSIX shell.
-Olaf.
--
___ Olaf 'Rhialto' Seibert -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl -- Cetero censeo "authored" delendum esse.
A: Because it messes up the order in which people normally read text.
>Q: Why is top-posting such a bad thing?
>>A: Top-posting.
>>>Q: What is the most annoying thing in mail and news?
James Vega
>
> On Tue 17 Jun 2008 at 10:44:50 -0400, James Vega wrote:
> > I have to agree with Chip here that the upstream version should default
> > to Bourne as that is most likely to cause people to actually configure
> > the highlighting to work properly for the variant of /bin/sh that *they*
> > are using.
>
> But the current Bourne shell *is* a POSIX shell.
You're thinking of the Bourne-Again SHell (aka bash). I'm referring to
the actual Bourne shell, which isn't obsolete yet.
I'll also reiterate that the current default has a purpose in being
annoyingly wrong in how it syntax highlights the code people using
modern systems are writing. It forces you to choose what the correct
syntax highlighting is when there is no other clear indication what
shell you are coding towards.
None of this would be a problem if you either use a proper shebang line
or let the proper flavor variable in your ~/.vimrc. It's better to be
consistently wrong than to inconsistently guess the wrong shell flavor
through various hacks.
Rhialto
> On Mon, Jun 23, 2008 at 09:47:15PM +0200, Rhialto wrote:
> >
> > On Tue 17 Jun 2008 at 10:44:50 -0400, James Vega wrote:
> > > I have to agree with Chip here that the upstream version should default
> > > to Bourne as that is most likely to cause people to actually configure
> > > the highlighting to work properly for the variant of /bin/sh that *they*
> > > are using.
> >
> > But the current Bourne shell *is* a POSIX shell.
>
> You're thinking of the Bourne-Again SHell (aka bash). I'm referring to
> the actual Bourne shell, which isn't obsolete yet.
No, I'm not. On the BSDs for instance, /bin/sh is not bash nor ksh nor
ash nor dash, but it *is* a POSIX shell. For the simple reason that
currenent Unixen ought to be POSIX compliant.
I have appended a few fragments from sh(1) at the end.
Or see
http://www.freebsd.org/cgi/man.cgi?query=sh&apropos=0&sektion=0&manpath=FreeBSD+7.0-RELEASE&format=html
for historical manpages of FreeBSD and other BSDs. Even 4.4BSD Lite2
already mentions $() and $(()). In 4.3BSD Net/2 sh is ash and the
manpage mentions $() but not $(()).
> I'll also reiterate that the current default has a purpose in being
> annoyingly wrong in how it syntax highlights the code people using
> modern systems are writing. It forces you to choose what the correct
> syntax highlighting is when there is no other clear indication what
> shell you are coding towards.
I totally agree with that. I very much dislike the mentality (that I'd
like to call the "Linux mentality", much like the older "all the world
is a VAX mentality") of "if it works on my machine it must be ok".
But on current Unixen, sh is POSIX, and has been for many years, simply
because Unix itself is POSIX. I would think that on most currently
installed systems this is true, and certainly on all new installations.
Well, Solaris is an exception.
SH(1) NetBSD General Commands Manual SH(1)
NAME
sh -- command interpreter (shell)
SYNOPSIS
sh [-aCefnuvxIimqVEb] [+aCefnuvxIimqVEb] [-o option_name]
[+o option_name] [command_file [argument ...]]
sh -c [-aCefnuvxIimqVEb] [+aCefnuvxIimqVEb] [-o option_name]
[+o option_name] command_string [command_name [argument ...]]
sh -s [-aCefnuvxIimqVEb] [+aCefnuvxIimqVEb] [-o option_name]
[+o option_name] [argument ...]
DESCRIPTION
sh is the standard command interpreter for the system. The current ver-
sion of sh is in the process of being changed to conform with the POSIX
1003.2 and 1003.2a specifications for the shell. This version has many
features which make it appear similar in some respects to the Korn shell,
but it is not a Korn shell clone (see ksh(1)). Only features designated
by POSIX, plus a few Berkeley extensions, are being incorporated into
this shell. This man page is not intended to be a tutorial or a complete
specification of the shell.
...
Command Substitution
Command substitution allows the output of a command to be substituted in
place of the command name itself. Command substitution occurs when the
command is enclosed as follows:
$(command)
or (``backquoted'' version):
`command`
The shell expands the command substitution by executing command in a sub-
shell environment and replacing the command substitution with the stan-
dard output of the command, removing sequences of one or more <newline>s
at the end of the substitution. (Embedded <newline>s before the end of
the output are not removed; however, during field splitting, they may be
translated into <space>s, depending on the value of IFS and quoting that
is in effect.)
Arithmetic Expansion
Arithmetic expansion provides a mechanism for evaluating an arithmetic
expression and substituting its value. The format for arithmetic expan-
sion is as follows:
$((expression))
The expression is treated as if it were in double-quotes, except that a
double-quote inside the expression is not treated specially. The shell
expands all tokens in the expression for parameter expansion, command
substitution, and quote removal.
Next, the shell treats this as an arithmetic expression and substitutes
the value of the expression.
Arithmetic expressions use a syntax similar to that of the C language,
and are evaluated using the `intmax_t' data type (this is an extension to
POSIX, which requires only `long' arithmetic). Shell variables may be
referenced by name inside an arithmetic expression, without needing a
``$'' sign.
> James
Dominique Pelle
> On Mon 23 Jun 2008 at 16:14:51 -0400, James Vega wrote:
>> On Mon, Jun 23, 2008 at 09:47:15PM +0200, Rhialto wrote:
>> >
>> > On Tue 17 Jun 2008 at 10:44:50 -0400, James Vega wrote:
>> > > I have to agree with Chip here that the upstream version should default
>> > > to Bourne as that is most likely to cause people to actually configure
>> > > the highlighting to work properly for the variant of /bin/sh that *they*
>> > > are using.
>> >
>> > But the current Bourne shell *is* a POSIX shell.
>>
>> You're thinking of the Bourne-Again SHell (aka bash). I'm referring to
>> the actual Bourne shell, which isn't obsolete yet.
>
> No, I'm not. On the BSDs for instance, /bin/sh is not bash nor ksh nor
> ash nor dash, but it *is* a POSIX shell. For the simple reason that
> currenent Unixen ought to be POSIX compliant.
I remember programming for the Bourne Shell on HPUX years ago,
and /bin/sh certainly did not have $(...) but only had backticks `...`.
I did a quick, search and found this which confirms it:
http://docs.hp.com/en/B2355-90046/ch15s03.html
== BEGIN QUOTE ===
The POSIX and Korn Shells are supersets of the Bourne Shell and
contain all of the Bourne Shell's syntactic constructs, and almost all
of its semantics. However, POSIX and Korn Shell implement some
features above and beyond those in the Bourne shell. They are:
...snip...
Command substitution syntax $(command).
...snip...
== END QUOTE ===
So it's fine if $(...) is highlighted as an error by default in my opinion.
If you don't want it as an error, you can add this to your ~/.vimrc:
let g:is_posix=1
-- Dominique
Michael Wojcik
> On Mon 23 Jun 2008 at 16:14:51 -0400, James Vega wrote:
>> On Mon, Jun 23, 2008 at 09:47:15PM +0200, Rhialto wrote:
>>> But the current Bourne shell *is* a POSIX shell.
>> You're thinking of the Bourne-Again SHell (aka bash). I'm referring to
>> the actual Bourne shell, which isn't obsolete yet.
>
> No, I'm not. On the BSDs for instance, /bin/sh is not bash nor ksh nor
> ash nor dash, but it *is* a POSIX shell. For the simple reason that
> currenent Unixen ought to be POSIX compliant.
Rhialto is correct that the current Unix shell is a POSIX shell, for
the simple reason that the Single Unix Specification unified UNIX,
POSIX (IEEE 1003), and ISO/IEC 9945. See the frontmatter of the Open
Group Base Specifications Issue 6, available from opengroup.org.
/bin/sh on a conforming Unix system is an Open Group / POSIX / ISO
shell. Whether it could properly be called the "Bourne shell" is
perhaps debatable, since it's no longer a "traditional" Bourne shell
with just the behavior and syntax of Steve Bourne's shell from Unix
7th Edition. But the current Unix shell is definitely a descendant of
the Bourne shell, and closer to it than to bash or ksh, much less the
csh family or more exotic shells.
And yes, it does support $() for command substitution.
> I have appended a few fragments from sh(1) at the end.
The BSD man pages are hardly authoritive, however. Unix is defined by
its standard, and its standard says the shell supports $().
--
Michael Wojcik
Micro Focus
Rhetoric & Writing, Michigan State University