[vim/vim] Syntax highlighting for Vim script comments is broken (Issue #11307)
chdiza
Steps to reproduce
Make a buffer with &ft=vim and syntax highlighting on. Make a comment line that also has an additional double-quote in it later, e.g.,:
" This is what we call " blahExpected behaviour
That entire line should be colored as a comment. But only the part from the second quote onward is colored that way. It looks like "This is what we call has the highlight group "vimIsCommand".
This is a bad regression; Vim didn't formerly do this. It's common to have lines that have a trailing comment, and then to comment out those very lines later. In my colorschemes, this makes the "This is what we call part colored the same as the "Normal" group; it defeats the purpose of syntax highlighting if one can't tell comments from non-comments by color.
Bisection reveals that the first bad commit is the following recent runtime file update:
f269eabc6c4f5bdcef989cd5b4b95ba8ccaa4d8a is the first bad commit
commit f269eabc6c4f5bdcef989cd5b4b95ba8ccaa4d8a
Author: Bram Moolenaar <Br...@vim.org>
Date: Mon Oct 3 18:04:35 2022 +0100
Update runtime files
Version of Vim
9.0.688
Environment
Ubuntu 20.04, though it also happens on macOS.
Terminal is kitty, or uxterm, doesn't matter
Also happens in gVim.
Logs and stack traces
No response
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Bram Moolenaar
> ### Steps to reproduce
>
> Make a buffer with &ft=vim and syntax highlighting on. Make a comment
> line that also has an additional double-quote in it later, e.g.,:
>
> " This is what we call " blah
>
> ### Expected behaviour
>
> That entire line should be colored as a comment. But only the part from the second quote onward is colored that way. It looks like `"This is what we call ` has the highlight group "vimIsCommand".
>
> This is a bad regression; Vim didn't formerly do this. It's common to
> have lines that have a trailing comment, and then to comment out those
> very lines later. In my colorschemes, this makes the `"This is what
> we call ` part colored the same as the "Normal" group; it defeats the
> purpose of syntax highlighting if one can't tell comments from
> non-comments by color.
just like a comment in legacy script. It's very hard to tell them
apart. For the time being, an unmatched quote in a legacy comment is
considered uncommon, therefore the highlighting as it is now.
A proper solution would know whether the line is in legacy script or
Vim9 script, which currently isn't done.
--
CONCORDE: Message for you, sir.
He falls forward revealing the arrow with the note.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
chdiza
A proper solution would know whether the line is in legacy script or Vim9 script,
I don't understand. Surely vim is already aware that it's highlighting a legacy comment---otherwise it wouldn't make any of my example line colored like a comment, because there are no '#' characters in it.
Plus, in the commit mentioned above, the line of vim.vim that got changed and broke things has got nothing to do with vim9script, AFAICT.
Paging Dr. @cecamp.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Shane-XB-Qian
i also thought for now such highlight for comment (for such case) is bad, which caused hard to find it was a comment.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Shane-XB-Qian
vim9 with # looks ok vs but legacy with " behavior different now at least.
// it thought whole line is a comment when vim9 with #.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Martin Tournoij
A proper solution would know whether the line is in legacy script or Vim9 script,
I don't understand. Surely vim is already aware that it's highlighting a legacy comment---otherwise it wouldn't make any of my example line colored like a comment, because there are no '#' characters in it.
Plus, in the commit mentioned above, the line of vim.vim that got changed and broke things has got nothing to do with vim9script, AFAICT.
The issue is that this:
vim9script
var x = "hello"
is valid in Vim9Script, it assigns a string to the variable x, but in VimScript the " is interpreted as a comment.
Disambiguating between the two uses of " is pretty hard.
That patch changed vimLineComment from +^[ \t:]*".*$+ to +^[ \t:]*"\("[^"]*"\|[^"]\)*$+. I'm not sure on the context of that change (i.e. what it fixed), but it's probably something that got highlighted as a comment when it shouldn't have.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
chdiza
In that example, Vim knows what to do because vim9script is present in that buffer. I don't use vim9script, so no such line is in any of my buffers, so Vim should know that it's dealing with legacy script.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Martin Tournoij
In that example, Vim knows what to do because
vim9scriptis present in that buffer. I don't use vim9script, so no such line is in any of my buffers, so Vim should know that it's dealing with legacy script.
You can use fun in vim9script and def in VimScript. You can mix and match the two.
Clearly things could be improved, but it's not that simple.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Bram Moolenaar
> > In that example, Vim knows what to do because `vim9script` is
> > present in that buffer. I don't use vim9script, so no such line is
> > in any of my buffers, so Vim should know that it's dealing with
> > legacy script.
>
> You can use `fun` in vim9script and `def` in VimScript. You can mix
> and match the two.
>
> Clearly things could be improved, but it's not that simple.
give it a try.
The specific change is a quick fix to tackle:
var foo =
"text"
This was highlighting "text" as a comment, which also caused trouble for
indenting. It's not perfect, but should work in most common situations.
--
ARTHUR: Well, I can't just call you `Man'.
DENNIS: Well, you could say `Dennis'.
ARTHUR: Well, I didn't know you were called `Dennis.'
DENNIS: Well, you didn't bother to find out, did you?
The Quest for the Holy Grail (Monty Python)
/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
chdiza
You can use fun in vim9script and def in VimScript. You can mix and match the two.
But you can't mix and match using the double-quote for comments. So what's wrong with making the vim syntax file work like this: if there is already a line that says "vim9script", then handle the quotes one way; otherwise, handle them the normal legacy way?
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Doug Kearns
Legacy comments are required in legacy functions in a Vim9 script file.
Ignoring the wider fix, there's already a shared FuncBody syntax region for function blocks. It probably just needs to be split into two and each limited to including the appropriate comment type.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Doug Kearns
Here's a very quick attempt at fixing this. The general approach is that only the correct comments for the script version are allowed at TOP level and the rest are contained. This means that some inappropriate comments will still be highlighted but they would generally be invalid syntax.
This certainly needs more work and the comment highlighting could be simplified generally. E.g., vim9Comment appears to be unnecessarily contained in several syntax groups where vimComment was only included to prevent strings from being erroneously highlighted.
diff --git a/runtime/syntax/vim.vim b/runtime/syntax/vim.vim index 839ae163f..3f4a351b6 100644 --- a/runtime/syntax/vim.vim +++ b/runtime/syntax/vim.vim @@ -243,14 +243,16 @@ endif " Functions : Tag is provided for those who wish to highlight tagged functions {{{2 " ========= syn cluster vimFuncList contains=vimCommand,vimFunctionError,vimFuncKey,Tag,vimFuncSID -syn cluster vimFuncBodyList contains=vimAbb,vimAddress,vimAugroupKey,vimAutoCmd,vimCmplxRepeat,vimComment,vim9Comment,vimContinue,vimCtrlChar,vimEcho,vimEchoHL,vimEnvvar,vimExecute,vimIsCommand,vimFBVar,vimFunc,vimFunction,vimFuncVar,vimGlobal,vimHighlight,vimIsCommand,vimLet,vimLetHereDoc,vimLineComment,vimMap,vimMark,vimNorm,vimNotation,vimNotFunc,vimNumber,vimOper,vimOperParen,vimRegion,vimRegister,vimSearch,vimSet,vimSpecFile,vimString,vimSubst,vimSynLine,vimUnmap,vimUserCommand -syn match vimFunction "\<\(fu\%[nction]\)!\=\s\+\%(<[sS][iI][dD]>\|[sSgGbBwWtTlL]:\)\=\%(\i\|[#.]\|{.\{-1,}}\)*\ze\s*(" contains=@vimFuncList nextgroup=vimFuncBody -syn match vimFunction "\<def!\=\ze\s*(" contains=@vimFuncList nextgroup=vimFuncBody +syn cluster vimFuncBodyList contains=vimAbb,vimAddress,vimAugroupKey,vimAutoCmd,vimCmplxRepeat,vimContinue,vimCtrlChar,vimEcho,vimEchoHL,vimEnvvar,vimExecute,vimIsCommand,vimFBVar,vimFunc,vimFunction,vimFuncVar,vimGlobal,vimHighlight,vimIsCommand,vimLet,vimLetHereDoc,vimMap,vimMark,vimNorm,vimNotation,vimNotFunc,vimNumber,vimOper,vimOperParen,vimRegion,vimRegister,vimSearch,vimSet,vimSpecFile,vimString,vimSubst,vimSynLine,vimUnmap,vimUserCommand +syn match vimFunction "\<\(fu\%[nction]\)!\=\s\+\%(<[sS][iI][dD]>\|[sg]:\)\=\%(\i\|[#.]\|{.\{-1,}}\)\+\ze\s*(" contains=@vimFuncList nextgroup=vimFuncBody +syn match vimFunction "\<def!\=\s\+\%(<[sS][iI][dD]>\|[sg]:\)\=\%(\i\|[#.]\|{.\{-1,}}\)\+\ze\s*(" contains=@vimFuncList nextgroup=vim9FuncBody if exists("g:vimsyn_folding") && g:vimsyn_folding =~# 'f' - syn region vimFuncBody contained fold start="\ze\s*(" matchgroup=vimCommand end="\<\(endf\>\|endfu\%[nction]\>\|enddef\>\)" contains=@vimFuncBodyList + syn region vimFuncBody contained fold start="\ze\s*(" matchgroup=vimCommand end="\<endfu\%[nction]\>" contains=@vimFuncBodyList,vimComment,vimLineComment + syn region vim9FuncBody contained fold start="\ze\s*(" matchgroup=vimCommand end="\<enddef\>" contains=@vimFuncBodyList,vim9Comment,vim9LineComment else - syn region vimFuncBody contained start="\ze\s*(" matchgroup=vimCommand end="\<\(endf\>\|endfu\%[nction]\>\|enddef\>\)" contains=@vimFuncBodyList + syn region vimFuncBody contained start="\ze\s*(" matchgroup=vimCommand end="\<endfu\%[nction]\>" contains=@vimFuncBodyList,vimComment,vimLineComment + syn region vim9FuncBody contained start="\ze\s*(" matchgroup=vimCommand end="\<enddef\>" contains=@vimFuncBodyList,vim9Comment,vim9LineComment endif syn match vimFuncVar contained "a:\(\K\k*\|\d\+\)" syn match vimFuncSID contained "\c<sid>\|\<s:" @@ -301,18 +303,34 @@ syn match vimUserAttrbCmplt contained "custom,\u\w*" " Lower Priority Comments: after some vim commands... {{{2 " ======================= -syn match vimComment excludenl +\s"[^\-:.%#=*].*$+lc=1 contains=@vimCommentGroup,vimCommentString -syn match vimComment +\<endif\s\+".*$+lc=5 contains=@vimCommentGroup,vimCommentString -syn match vimComment +\<else\s\+".*$+lc=4 contains=@vimCommentGroup,vimCommentString -syn region vimCommentString contained oneline start='\S\s\+"'ms=e end='"' -" Vim9 comments - TODO: might be highlighted while they don't work -syn match vim9Comment excludenl +\s#[^{].*$+lc=1 contains=@vimCommentGroup,vimCommentString -syn match vim9Comment +\<endif\s\+#[^{].*$+lc=5 contains=@vimCommentGroup,vimCommentString -syn match vim9Comment +\<else\s\+#[^{].*$+lc=4 contains=@vimCommentGroup,vimCommentString -" Vim9 comment inside expression -syn match vim9Comment +\s\zs#[^{].*$+ms=s+1 contains=@vimCommentGroup,vimCommentString -syn match vim9Comment +^\s*#[^{].*$+ contains=@vimCommentGroup,vimCommentString -syn match vim9Comment +^\s*#$+ contains=@vimCommentGroup,vimCommentString +let s:isVim9Script = "\n" .. getline(1, 10)->join("\n") =~# '\n\s*vim9\%[script]\>' +if s:isVim9Script + syn match vimComment excludenl +\s"[^\-:.%#=*].*$+lc=1 contains=@vimCommentGroup,vimCommentString contained + syn match vimComment +\<endif\s\+".*$+lc=5 contains=@vimCommentGroup,vimCommentString contained + syn match vimComment +\<else\s\+".*$+lc=4 contains=@vimCommentGroup,vimCommentString contained + syn region vimCommentString contained oneline start='\S\s\+"'ms=e end='"' contained + " Vim9 comments - TODO: might be highlighted while they don't work + syn match vim9Comment excludenl +\s#[^{].*$+lc=1 contains=@vimCommentGroup,vimCommentString + syn match vim9Comment +\<endif\s\+#[^{].*$+lc=5 contains=@vimCommentGroup,vimCommentString + syn match vim9Comment +\<else\s\+#[^{].*$+lc=4 contains=@vimCommentGroup,vimCommentString + " Vim9 comment inside expression + syn match vim9Comment +\s\zs#[^{].*$+ms=s+1 contains=@vimCommentGroup,vimCommentString + syn match vim9Comment +^\s*#[^{].*$+ contains=@vimCommentGroup,vimCommentString + syn match vim9Comment +^\s*#$+ contains=@vimCommentGroup,vimCommentString +else + syn match vimComment excludenl +\s"[^\-:.%#=*].*$+lc=1 contains=@vimCommentGroup,vimCommentString + syn match vimComment +\<endif\s\+".*$+lc=5 contains=@vimCommentGroup,vimCommentString + syn match vimComment +\<else\s\+".*$+lc=4 contains=@vimCommentGroup,vimCommentString + syn region vimCommentString contained oneline start='\S\s\+"'ms=e end='"' + " Vim9 comments - TODO: might be highlighted while they don't work + syn match vim9Comment excludenl +\s#[^{].*$+lc=1 contains=@vimCommentGroup,vimCommentString contained + syn match vim9Comment +\<endif\s\+#[^{].*$+lc=5 contains=@vimCommentGroup,vimCommentString contained + syn match vim9Comment +\<else\s\+#[^{].*$+lc=4 contains=@vimCommentGroup,vimCommentString contained + " Vim9 comment inside expression + syn match vim9Comment +\s\zs#[^{].*$+ms=s+1 contains=@vimCommentGroup,vimCommentString contained + syn match vim9Comment +^\s*#[^{].*$+ contains=@vimCommentGroup,vimCommentString contained + syn match vim9Comment +^\s*#$+ contains=@vimCommentGroup,vimCommentString contained +endif " Environment Variables: {{{2 " ===================== @@ -649,8 +667,13 @@ syn match vimCtrlChar "[^A-^H^K^O-^_]" " Beginners - Patterns that involve ^ {{{2 " ========= -syn match vimLineComment +^[ \t:]*"\("[^"]*"\|[^"]\)*$+ contains=@vimCommentGroup,vimCommentString,vimCommentTitle -syn match vim9LineComment +^[ \t:]\+#.*$+ contains=@vimCommentGroup,vimCommentString,vimCommentTitle +if s:isVim9Script + syn match vimLineComment +^[ \t:]*".*$+ contains=@vimCommentGroup,vimCommentString,vimCommentTitle contained + syn match vim9LineComment +^[ \t:]\+#.*$+ contains=@vimCommentGroup,vimCommentString,vimCommentTitle +else + syn match vimLineComment +^[ \t:]*".*$+ contains=@vimCommentGroup,vimCommentString,vimCommentTitle + syn match vim9LineComment +^[ \t:]\+#.*$+ contains=@vimCommentGroup,vimCommentString,vimCommentTitle contained +endif syn match vimCommentTitle '"\s*\%([sS]:\|\h\w*#\)\=\u\w*\(\s\+\u\w*\)*:'hs=s+1 contained contains=vimCommentTitleLeader,vimTodo,@vimCommentGroup syn match vimContinue "^\s*\\" syn region vimString start="^\s*\\\z(['"]\)" skip='\\\\\|\\\z1' end="\z1" oneline keepend contains=@vimStringGroup,vimContinue
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Enrico Maria De Angelis
I wonder if it is related to #11458.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Luke Davis
For anyone coming across this thread looking for a quick drop-in .vimrc fix (as I was), and who hasn't transitioned to vim9 script yet, the following seems to restore original syntax highlighting behavior:
function! s:syntax_overrides() abort syntax match customComment /^[ \t:]*".*$/ contains=vimCommentTitle,vimCommentString,@vimCommentGroup highlight link customComment vimComment endfunction augroup syntax_overrides au! au FileType vim call s:syntax_overrides() augroup END
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
dkearns
Closing as I believe everything discussed in this issue is now highlighted properly.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
dkearns
Closed #11307 as completed.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.