New syntax file for shell scripts

[Radek]

Hi there,

I didn't like how the current highlighting for shell scripts behaved in
certain situations, so I wrote my own syntax file. It supports
POSIX-compliant (POSIX.1-2008) shell scripts and Bash (4.2.10).

It's designed to be more syntax-driven than the current one (e.g. it
does not highlight every keyword it finds as a command name - only
those that are actual command names). As a side effect, it can catch
many invalid constructs and highlight them as errors.

I encourage all of you that do some shell scripting to try it out
and see if you like it. Comments and suggestions are welcome. Enjoy! :)

I also include the colorscheme that I use ("radek.vim") - it's intended
mostly for GUI version.

[Thilo Six]

Radek wrote the following on 21.05.2011 03:29

I am certain that it´s better to use

setlocal iskeyword
^^^^^

I will try it out.
--
bye Thilo

4096R/0xC70B1A8F
721B 1BA0 095C 1ABA 3FC6 7C18 89A4 A2A0 C70B 1A8F

[Lech Lorens]

On 21-May-2011 Radek <truer...@o2.pl> wrote:
> Hi there,
>
> I didn't like how the current highlighting for shell scripts behaved in
> certain situations, so I wrote my own syntax file. It supports
> POSIX-compliant (POSIX.1-2008) shell scripts and Bash (4.2.10).
>
> It's designed to be more syntax-driven than the current one (e.g. it
> does not highlight every keyword it finds as a command name - only
> those that are actual command names). As a side effect, it can catch
> many invalid constructs and highlight them as errors.

The following fragments in my scripts get highlighted as errors:

#v+
#!/bin/bash

if [[ "$1" = "--offset" ]] ; then
true
elif [[ "$1" =~ --offset=(.*) ]] ; then
false
else
true;
false;
fi
#v-

(parentheses and the following space).

#v+
#!/bin/bash

for(( ; $# > 0; ))
do
true
done
#v-

(beginning with '$#' up to 'done').

#v+
#!/usr/bin/env bash

original=$6
modified=${original%.svn-base}
modified=${modified/\/.svn\/text-base/}
#v-

('\/.svn\/text-base/').

What's strange is that sometimes the last one gets highlighted
correctly. It suffices to execute "setf sh" once again.

--
Cheers,
Lech

[Radek]

On Sat, 21 May 2011 15:08:51 +0200
Lech Lorens <lech....@gmail.com> wrote:

> The following fragments in my scripts get highlighted as errors:
>
> #v+
> #!/bin/bash
>
> if [[ "$1" = "--offset" ]] ; then
> true
> elif [[ "$1" =~ --offset=(.*) ]] ; then
> false
> else
> true;
> false;
> fi
> #v-
>
> (parentheses and the following space).

Regex highlighting in bash conditionals isn't implemented yet, but I'll
add it soon. Right now it highlights them as plain shell words, so any
unquoted special character (like parens) triggers an error.

>
>
> #v+
> #!/bin/bash
>
> for(( ; $# > 0; ))
> do
> true
> done
> #v-
>
> (beginning with '$#' up to 'done').

As above, 'for' loop with arithmetic expressions isn't yet there (I just
forgot about it, will do it soon - it's a matter of adding few lines).

> #v+
> #!/usr/bin/env bash
>
> original=$6
> modified=${original%.svn-base}
> modified=${modified/\/.svn\/text-base/}
> #v-
>
> ('\/.svn\/text-base/').
>
> What's strange is that sometimes the last one gets highlighted
> correctly. It suffices to execute "setf sh" once again.

When it highlights the mentioned fragment as error, check if
'b:is_bash' variable is set. If it's set, it should highlight properly,
and if it's not set, then it should detect it as invalid syntax
(because it's bash-specific).

Thanks for feedback :)

[Radek]

Updated version. Changes:
* refactored 'for' loop - no functional change, just simpler code
* added arithmetic variant of 'for' loop
* set 'iskeyword' option using :setlocal, as Thilo suggested (thanks!)

Now, it seems that only regexes in Bash '=~' conditional remain for the
syntax to be complete.

[Radek Nadstawny]

Another update:

* Reworked highlighting groups - it's now much easier to customize
which highlighting groups are used by various kinds of syntax items
Example: you can make closing quotes highlight differently by only
adding "hi link shClosingQuote <desired highlight group>" to your
vimrc

* Added proper regex highlighting in Bash' compound conditionals

* Added Bash' "time" keyword

* Fixed few bugs

How do you like it now? This time it should correctly highlight any
valid script, the only exceptions being those few cases listed in
"Known Bugs" section at the bottom of the file.

[Jean-Rene David]

* Radek Nadstawny [2011.05.23 22:30]:
[...]

> * Added proper regex highlighting in Bash' compound conditionals
>
> * Added Bash' "time" keyword

If this syntax file to include bash-specific constructs, then IMHO it
should be called bash.vim, not sh.vim.

--
JR

[Gary Johnson]

The current $VIMRUNTIME/syntax/sh.vim also has bash-specific and
even ksh-specific constructs, so the name is consistent with current
practice. On the other hand, if this sh.vim is to replace
$VIMRUNTIME/syntax/sh.vim, it should also support ksh constructs.

Regards,
Gary

[Gary Johnson]

On 2011-05-24, Radek Nadstawny wrote:

> I don't use ksh, but I know that some parts of POSIX as well as many
> Bash features were inspired by ksh, so it's likely that making this
> syntax ksh-compatible will be fairly easy. I would be honored if my
> sh.vim made it into the official Vim distribution, so I can look into
> it ;)

I no longer use ksh either, but it is the officially-supported shell
where I used to work, so I imagine that some Vim users continue to
use it and will expect it to be supported.

Thanks for your work on sh.vim. I've installed it but I don't use
it very often, so I won't be good source of feedback.

Regards,
Gary

[Radek Nadstawny]

On Tue, 24 May 2011 07:23:08 -0700
Gary Johnson <gary...@spocom.com> wrote:

> The current $VIMRUNTIME/syntax/sh.vim also has bash-specific and
> even ksh-specific constructs, so the name is consistent with current
> practice. On the other hand, if this sh.vim is to replace
> $VIMRUNTIME/syntax/sh.vim, it should also support ksh constructs.

Basically you're right. If Vim doesn't detect a file as a Bash script,
my syntax defaults to POSIX-compliance mode. If Vim determines a file
to be a Bash script (e.g. its first line is "#!/bin/bash"), or if the
user has g:is_bash variable set, then (and only then) Bash-specific
constructs are enabled. If you use some ancient shells that don't
support POSIX features, it wouldn't be too hard for me to make the
default bare bones, without e.g. "${var##prefix}" or "$(...)" command
substitution and enable full POSIX only on certain conditions.

[sc]

On Tuesday, May 24, 2011 13:43:52 Gary Johnson wrote:

> Thanks for your work on sh.vim. I've installed it but I don't
> use it very often, so I won't be good source of feedback.

i'm using it, and so far so good ― will be glad to report back
if i find issues

sc

[George V. Reilly]

On Mon, May 23, 2011 at 7:22 PM, Radek Nadstawny <truer...@o2.pl> wrote:

Another update:

If I go to the beginning of a line such as this:

build_number=${build_number:-37} #### for standalone testing

and type 'ye', it yanks everything up to the '}'. With the stock syntax highlighting, it just yanks 'build_number'

-- 
/George V. Reilly  geo...@reilly.org  Twitter: @georgevreilly
http://www.georgevreilly.com/blog  http://blogs.cozi.com/tech

[Radek Nadstawny]

On Tue, 24 May 2011 17:07:57 -0700
"George V. Reilly" <geo...@reilly.org> wrote:

> If I go to the beginning of a line such as this:
>
> build_number=${build_number:-37} #### for standalone testing
>
> and type 'ye', it yanks everything up to the '}'. With the stock
> syntax highlighting, it just yanks 'build_number'

This is probably because I have set the 'iskeyword' option to what the
shell actually recognises as a word character - only whitespace and
special shell characters |&;<>() break words in shell scripts.

It's quite important to get this right - otherwise Vim would
misinterpret stuff, e.g. take "if[" as "if" compound followed by "["
test, while in fact it's just a single (valid!) command name.

In theory, it could be fixed by replacing every "\<" and "\>" in
patterns by equivalent that doesn't depend on 'iskeyword' value, but
that would be neither pretty nor maintainable. Is anyone aware of other
possible solutions?

[Radek Nadstawny]

Update, mostly bugfix

Bugs fixed:
* use of reserved word as a function name did not trigger an error
* [Bash] Arithmetic "for" variant was recognised only on top level.
(thanks to sc for testing and spotting the bug)
* [Bash] "time" keyword couldn't be used as a separate command
* [Bash] when "function" keyword was used, optional parentheses after
function name weren't recognised properly (also thanks to sc)
* [Bash] In parameter expansions of form "${var//pattern/replacement}"
the slash at the beginning of pattern was misinterpreted as end of
pattern

I'm not attaching the file directly, but you can reach it here:
http://dl.dropbox.com/u/30354453/sh.vim

Enjoy!

On side note, I'm working on ksh mode. I don't have ksh installed, but
I'm reading through ksh93 man page and it should give me all the
information I need.

[Ernie Rael]

Hi, I've been using sh.vim for a few weeks. I don't edit that many bash
files, so ... But I came across the following problem which didn't seem
to be in the list of known bugs. In the following, the end of the "-r)"
line to just before the "esac" is highlighted as errors. If there are
not multiple commands on the same line (don't use ';' to separate
commands) then everything is ok.

> #!/bin/bash
>
> while true
> do
> case "$1" in
> -r) rev[nRev++]=$2; shift 2;;
> -c) change[nChange]=$2; shift 2;;
> -f) nofork=1; shift;;
> -h) usage;;
> --) shift; break;;
> *) echo Internal Error; exit 1;;
> esac
> done

And thanks for the syntax file...
-ernie

[Ernie Rael]

And there's array assignment to a variable when declared local on the
same line
> #/usr/bin/bash
>
> local a=(x y)

On 5/25/2011 7:13 AM, Radek Nadstawny wrote: