[vim/vim] Fix, improve, and document Vim9 script EX_WHOLE and Ex command shortening issues (PR #20191)

3 views

Skip to first unread message

Peter Kenny

unread,

6:04 AM (6 hours ago) 6:04 AM

to vim/vim, Subscribed

This PR addresses many issues in the treatment of shortened commands. The initial aim was to address many missing instances of EX_WHOLE from commands in ex_cmds.h (including endclass, endenum, public, and others, which I raised as Issue 20032. The more I looked, the more there was to fix, improve or correct, including:

Additions to find_ex_command() in ex_docmd.c. The :horizontal minimum abbreviation is :hor, but :ho would be returned by fullcommand('ho'). :k is also fixed (it would return 'k' for any length, e.g., 'kaaaaargh').
Tests for the fixes in test_vimscript.vim and test_vim9_script.vim.
userfunc.c had enddef hardcoded, which meant E1065 errors for using endd or endde were outliers in that the "cannot be shortened" would not return the shortened, incorrect, command.
gen_syntax_vim.vim and vim.vim.base updates. There are too many things to list here, but a key overall improvement is treatment of legacy Vim script and Vim9 script for many commands is improved.
Syntax test files are extended and improved, including some things noted with "FIXME", which are fixed in the point above. New test files for shortened commands are added. This includes testing in .vim as well as .txt help files with Vim9 script code blocks. Lots of updated dumps.
Some corrections ('cont' is not the shortest form, for example) and fixing omissions related to shortened commands are made to six help files. There are some odd things included such as the different treatment of :dl between legacy Vim script and Vim9 script. A cross-reference to vim9-no-shorten is added in usr_20.txt, which is what sent me on this quest when documenting odd behaviours testing fullcommand() when updating Section 2 of vim9.txt (it can now be finished once these corrections are in place).
fullcommand() persisting limitations are flagged as TODO comments in the new tests.

There are some things that could be improved further, but I will create separate issues for those in future.

Testing (native Debian 13):
src: 7728 passing, 0 fail, and only 144 skipped n/a tests.
syntax: Test run on 2026 May 11 18:43:06, OK: 252, FAILED: 0: [], skipped: 0

You can view, comment on, or merge this pull request online at:

https://github.com/vim/vim/pull/20191

Commit Summary

ae8fd16 Fix, improve, and document Vim9 script EX_WHOLE and Ex command shortening

File Changes

(191 files)

Patch Links:

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.

h_east

unread,

9:11 AM (3 hours ago) 9:11 AM

to vim/vim, Subscribed

@h-east commented on this pull request.

Reviewed with the support of AI/LLM.

In runtime/syntax/generator/gen_syntax_vim.vim:

>  			\ v.name ==# 'horizontal' ?
 			\		extend(copy(v), {'omit_idx': 2, 'syn_str': 'hor[izontal]'}) :
 			\ v
 			\ })
 
+		" remove 'finally' - handle it separately in vim.vim.base

⬇️ Suggested change

-		" remove 'finally' - handle it separately in vim.vim.base 
+		" remove 'finally' - handle it separately in vim.vim.base

In runtime/syntax/generator/vim.vim.base:

> @@ -246,9 +246,9 @@ syn match	vimNumber	'\<0z\%(\x\x\)\+\%(\.\%(\x\x\)\+\)*'	skipwhite nextgroup=@vi
 syn case match
 
 " All vimCommands are contained by vimIsCommand. {{{2
-syn cluster vimCmdList	contains=vimAbb,vimAddress,vimAt,vimAutocmd,vimAugroup,vimBehave,vimBreakadd,vimBreakdel,vimBreaklist,vimCall,vimCatch,vimCd,vimCommandModifier,vimConst,vimDoautocmd,vimDebug,vimDebuggreedy,vimDef,vimDefFold,vimDefer,vimDelcommand,vimDelFunction,vimDoCommand,@vimEcho,vimElse,vimEnddef,vimEndfunction,vimEndif,vimEval,vimExecute,vimIsCommand,vimExtCmd,vimExFilter,vimExMark,vimFiletype,vimFor,vimFunction,vimFunctionFold,vimGrep,vimGrepAdd,vimGlobal,vimHelp,vimHelpgrep,vimHighlight,vimHistory,vimImport,vimLanguage,vimLet,vimLoadkeymap,vimLockvar,vimMake,vimMap,vimMark,vimMatch,vimNotFunc,vimNormal,vimProfdel,vimProfile,vimPrompt,vimRedir,vimSet,vimSleep,vimSort,vimSyntax,vimSyntime,vimSynColor,vimSynLink,vimTerminal,vimThrow,vimUniq,vimUnlet,vimUnlockvar,vimUnmap,vimUserCmd,vimVimgrep,vimVimgrepadd,vimWincmd,vimMenu,vimMenutranslate,@vim9CmdList,@vimExUserCmdList,vimLua,vimMzScheme,vimPerl,vimPython,vimPython3,vimPythonX,vimRuby,vimTcl
-syn cluster vim9CmdList	contains=vim9Abstract,vim9Class,vim9Const,vim9Enum,vim9Export,vim9Final,vim9For,vim9Interface,vim9Type,vim9Var
-syn match vimCmdSep	"\\\@1<!|"	skipwhite nextgroup=@vimCmdList,vimSubst1,@vimFunc
+syn cluster vimCmdList	contains=vimAbb,vimAddress,vimAt,vimAutocmd,vimAugroup,vimBehave,vimBreakadd,vimBreakdel,vimBreaklist,vimCall,vimCatch,vimCd,vimCommandModifier,vimConst,vimDoautocmd,vimDebug,vimDebuggreedy,vimDef,vimDefFold,vimDefer,vimDelcommand,vimDelFunction,vimDoCommand,@vimEcho,vimEnddef,vimEndfunction,vimEval,vimExecute,vimIsCommand,vimExtCmd,vimExFilter,vimExMark,vimFiletype,vimFor,vimFunction,vimFunctionFold,vimGrep,vimGrepAdd,vimGlobal,vimHelp,vimHelpgrep,vimHighlight,vimHistory,vimImport,vimLanguage,vimLet,vimLoadkeymap,vimLockvar,vimMake,vimMap,vimMark,vimMatch,vimNormal,vimProfdel,vimProfile,vimPrompt,vimRedir,vimSet,vimSleep,vimSort,vimSyntax,vimSyntime,vimSynColor,vimSynLink,vimTerminal,vimThrow,vimUniq,vimUnlet,vimUnlockvar,vimUnmap,vimUserCmd,vimVimgrep,vimVimgrepadd,vimWincmd,vimMenu,vimMenutranslate,@vim9CmdList,@vimExUserCmdList,vimLua,vimMzScheme,vimPerl,vimPython,vimPython3,vimPythonX,vimRuby,vimTcl,vimWholeFlow9
+syn cluster vim9CmdList	contains=vim9Abstract,vim9Catch,vim9Class,vim9Const,vim9Enum,vim9Export,vim9Final,vim9For,vim9Import,vim9Interface,vim9Throw,vim9Type,vim9Var,vim9WholeFlow9
+syn match vimCmdSep	"\\\@1<!|"	skipwhite nextgroup=@vimCmdList,vimSubst1,@vimFunc,vimInsert

vimCmdSep is defined unconditionally and applies to both legacy and Vim9
script. In Vim9 the Insertions section (around L289) replaces
append/change/insert with Normal highlight on purpose. If a Vim9
buffer contains something like | append, however, vimCmdSep will now
hand control over to vimInsert, which starts a region linked to
vimString. It would be worth double-checking in a real Vim9 buffer
whether this still matches the intent, and if not, guarding the addition
under if !s:vim9script.

In runtime/syntax/generator/vim.vim.base:

> @@ -756,7 +779,7 @@ if s:vim9script
   " super must be followed by '.'
   syn match	vim9Super		contained	"\.\@1<!\<super\.\@="
 
-  VimFoldc syn region	vim9ClassBody	start="\<class\>" matchgroup=vimCommand end="\<endclass\>" contains=@vim9ClassBodyList transparent
+  VimFoldc syn region	vim9ClassBody	start="\v:?<class>" matchgroup=vimCommand end="\v<endclass>" contains=@vim9ClassBodyList transparent

start regex with :? but end without it

What is the intent behind allowing an optional leading colon only on the
start side? :endclass / :endenum / :endinterface are equally valid,
so the asymmetry feels surprising. Since these are transparent regions
the practical impact may be nil, but matching the two sides would be
easier to read and reason about.

In runtime/syntax/generator/vim.vim.base:

> +  syn match	vim9Type		"\<type\>"	skipwhite nextgroup=vim9TypeAlias,vim9TypeAliasError
+"  syn keyword	vim9Type	type		skipwhite nextgroup=vim9TypeAlias,vim9TypeAliasError

commented-out alternative for vim9Type

This is not a leftover of the deleted line - it is an alternative
implementation written with syn keyword. If the intent is to keep it as
a reference, please add a one-liner above it (e.g. " Alternative form using syn keyword:) so the next reader understands it is intentional.

In runtime/syntax/generator/vim.vim.base:

> @@ -809,7 +832,7 @@ if s:vim9script
         \ skipwhite skipempty nextgroup=vim9EnumImplementedInterfaceComment,vim9EnumValue
         \ contains=@vimCommentGroup,vimCommentString
 
-  VimFolde syn region	vim9EnumBody	start="\<enum\>" matchgroup=vimCommand end="\<endenum\>" contains=@vim9EnumBodyList transparent
+  VimFolde syn region	vim9EnumBody	start="\v:?<enum>" matchgroup=vimCommand end="\v<endenum>" contains=@vim9EnumBodyList transparent

Same above.

In runtime/syntax/generator/vim.vim.base:

> @@ -836,17 +859,40 @@ if s:vim9script
         \ nextgroup=vim9AbstractDefParams
         \ contains=vim9DefTypeParam
 
-  VimFoldi syn region	vim9InterfaceBody	start="\<interface\>" matchgroup=vimCommand end="\<endinterface\>" contains=@vim9InterfaceBodyList transparent
+  VimFoldi syn region	vim9InterfaceBody	start="\v:?<interface>" matchgroup=vimCommand end="\v<endinterface>" contains=@vim9InterfaceBodyList transparent

Same above.

In runtime/syntax/generator/vim.vim.base:

> @@ -1478,7 +1530,9 @@ syn region	vimHelpgrepPattern	contained
 
 " Vimgrep: {{{2
 " =======
-syn match	vimVimgrep		"\<l\=vim\%[grep]\>"	skipwhite nextgroup=vimVimgrepBang,vimVimgrepPattern
+syn match	vimVimgrep		"\<lv\%[imgrep]\>"	skipwhite nextgroup=vimVimgrepBang,vimVimgrepPattern
+syn match	vimVimgrep		"\<vimg\%[rep]\>"		skipwhite nextgroup=vimVimgrepBang,vimVimgrepPattern
+" syn match	vimVimgrep		"\<l\=vim\%[grep]\>"	skipwhite nextgroup=vimVimgrepBang,vimVimgrepPattern

Same above.

In runtime/syntax/generator/vim.vim.base:

> @@ -1467,7 +1517,9 @@ syn region	vimHelpArg	contained
 syn match	vimHelpNextCommand	contained	"\ze|[^|]"	skipwhite nextgroup=vimCmdSep
 syn match	vimHelpBang		contained	"\a\@1<=!"	skipwhite nextgroup=vimHelpArg,vimHelpNextCommand
 
-syn match	vimHelpgrep		"\<l\=helpg\%[rep]\>"	skipwhite nextgroup=vimHelpgrepBang,vimHelpgrepPattern
+syn match	vimHelpgrep		"\<lh\%[elpgrep]\>"	skipwhite nextgroup=vimHelpgrepBang,vimHelpgrepPattern
+syn match	vimHelpgrep		"\<helpg\%[rep]\>"	skipwhite nextgroup=vimHelpgrepBang,vimHelpgrepPattern
+" syn match	vimHelpgrep		"\<l\=helpg\%[rep]\>"	skipwhite nextgroup=vimHelpgrepBang,vimHelpgrepPattern

commented-out previous regex

These look like the regexes that were just removed, kept commented out.
Git history already preserves them, so removing the comments would keep
the file cleaner unless there is a specific reason to retain them.

In runtime/syntax/generator/vim.vim.base:

> +  syn match	vimAugroupName	contained	"\%(\\["|[:space:]]\|[^"|[:space:]]\)\+"
+      \                                                                 skipwhite nextgroup=vimCmdSep,vim9Comment
+  syn match	vimAugroupEnd	contained	"\c\<END\>"	skipwhite nextgroup=vimCmdSep,vim9Comment

Duplication between Vim9 and legacy vimAugroup*

The only difference between the two branches is the comment highlight name
(vim9Comment vs vimComment). Extracting that into a local variable
would let the two syn match calls live outside the if block. This is
a stylistic suggestion; the generator file may have its own conventions
that justify the current shape.

In runtime/syntax/generator/vim.vim.base:

>        \                                                                 skipwhite nextgroup=vimCmdSep,vimComment
-syn match	vimAugroupEnd	contained	"\c\<END\>"	skipwhite nextgroup=vimCmdSep,vimComment
+  syn match	vimAugroupEnd	contained	"\c\<END\>"	skipwhite nextgroup=vimCmdSep,vimComment

Same above.

In runtime/syntax/generator/vim.vim.base:

> -syn match	vimDef	"\<def\>"		skipwhite nextgroup=vimDefBang,vimDefName,vimFunctionPattern,vimCmdSep,vimComment
+syn match	vimDef	"\<def\>"		skipwhite nextgroup=vimDefBang,vimDefName,vimFunctionPattern,vimCmdSep,vim9Comment

vimDef nextgroup changed to vim9Comment

def is Vim9-only, so the change is reasonable. However, vimDef
itself is defined outside any s:vim9script guard. It would be worth
checking that a comment written after def in a legacy file (which is a
runtime error but still parsable) is still highlighted as intended.

In runtime/syntax/generator/vim.vim.base:

> +" :t		regular vim	co\%[py]
+" :let		Declarations	vimLet
+" :s? and s?? (:s flag variants)	two and three l	vimSubst
+"   Note these are valid but are different in Vim9 script:
+"   :sc (:scriptnames) :si (:simalt) :sr (:srewind)
+" :dp, :o[pen], :x[it] - no special Vim script handling
+if s:vim9script
+  " Normal is used for these because otherwise they will appear as errors when
+  " initially typing variables, e.g., '  t = true', '  sce = 9', etc.
+  syn match	Normal	"\%#=1\v%(%(^[:[:blank:]]*)|[[:blank:]]+[\x7c][:[:blank:]]*)%(%([[:digit:]]+%(,[[:digit:]]+)?)|[.$]|'[[:alpha:][\]<>'"^.(){}/&]|'[/][^/]*[/]|'[?][^?]*[?])?%(t|dp|k|let)>%([[:blank:]]+|$)"
+  syn match	Normal	"\v%(^[:[:blank:]]*|[[:blank:]]+[|][:[:blank:]]*)%(mod%[e]?|x%[it])\s*$"
+  syn match	Normal	"\%#=1\v%(%(^[:[:blank:]]*)|[[:blank:]]+[\x7c][:[:blank:]]*)%(%([[:digit:]]+%(,[[:digit:]]+)?)|[.$]|'[[:alpha:][\]<>'"^.(){}/&]|'[/][^/]*[/]|'[?][^?]*[?])?o%[pen]%([[:blank:]]+|$)"
+  " All substitute flag variants are invalid in Vim9 script except:
+  syn match	Normal	"\%#=1\v%(%(^[:[:blank:]]*)|[[:blank:]]+[\x7c][:[:blank:]]*)%(sce|scg|sci|scI|scl|scn|scp|sg|sgc|sge|sgi|sgI|sgl|sgn|sgp|sgr|sic|sie|siI|sin|sip|sir|sI|sIc|sIe|sIg|sIi|sIl|sIn|sIp|sIr|src|srg|sri|srI|srl|srn|srp)>[[:blank:]]*$"
+endif
+

The intent of hiding :t/:dp/:k/:let/:mod/:xit/:open and the
substitute flag variants under Normal highlight is described in the
section comment. The individual regexes are dense, though. Adding a
short example next to each (e.g. " matches: ' t = true', '| k = 1')
would make it considerably easier for a future maintainer to verify or
extend them.

In runtime/syntax/generator/vim.vim.base:

> +  " (All vimWholeFlow9 need to be keyword so that commands are matched
+  " correctly in a help filetype's >vim code block)
+  syn keyword	vimWholeFlow9	if	skipwhite nextgroup=@vimExprList,vimNotation
+  syn keyword	vimWholeFlow9	elsei[f]	skipwhite nextgroup=@vimExprList,vimNotation
+  syn keyword	vimWholeFlow9	retu[rn]	skipwhite nextgroup=@vimExprList,vimNotation
+  syn keyword	vimWholeFlow9	wh[ile]	skipwhite nextgroup=@vimExprList,vimNotation
+  syn keyword	vimWholeFlow9	brea[k]	skipwhite nextgroup=vimComment
+  syn keyword	vimWholeFlow9	con[tinue]	skipwhite nextgroup=vimComment
+  syn keyword	vimWholeFlow9	el[se]	skipwhite nextgroup=vimComment
+  syn keyword	vimWholeFlow9	en[dif]	skipwhite nextgroup=vimComment
+  syn keyword	vimWholeFlow9	endfo[r]	skipwhite nextgroup=vimComment
+  syn keyword	vimWholeFlow9	endt[ry]	skipwhite nextgroup=vimComment
+  syn keyword	vimWholeFlow9	endw[hile]	skipwhite nextgroup=vimComment
+  syn keyword	vimWholeFlow9	fina[lly]	skipwhite nextgroup=vimComment
+  syn keyword	vimWholeFlow9	fini[sh]	skipwhite nextgroup=vimComment
+endif

Naming of vim9WholeFlow9 / vimWholeFlow9

The trailing 9 appears twice in vim9WholeFlow9, and the legacy
counterpart vimWholeFlow9 also carries a 9. Which 9 corresponds to
Vim9 and which to EX_WHOLE is not obvious at a glance. A flatter name
such as vim9FlowKeyword / vimFlowKeyword would convey the role more
directly. Purely a naming preference.

In runtime/syntax/generator/vim.vim.base:

> +    syn match Normal	"^[: \t]*\(\d\+\(,\d\+\)\?\)\?a\%[ppend]\s*$"
+    syn match Normal	"^[: \t]*\(\d\+\(,\d\+\)\?\)\?c\%[hange]\s*$"
+    syn match Normal	"^[: \t]*\(\d\+\(,\d\+\)\?\)\?i\%[nsert]\s*$"

The leading ^[: \t]*$\d\+\(,\d\+$\?\)\? and trailing \s*$ are
identical in all three patterns; only the command token differs.
Merging them into a single rule with an alternation removes the
duplication and lets the regex engine evaluate the line once instead of
three times

⬇️ Suggested change

-    syn match Normal	"^[: \t]*\(\d\+\(,\d\+\)\?\)\?a\%[ppend]\s*$"
-    syn match Normal	"^[: \t]*\(\d\+\(,\d\+\)\?\)\?c\%[hange]\s*$"
-    syn match Normal	"^[: \t]*\(\d\+\(,\d\+\)\?\)\?i\%[nsert]\s*$"
+    syn match Normal	"^[: \t]*\(\d\+\(,\d\+\)\?\)\?\%(a\%[ppend]\|c\%[hange]\|i\%[nsert]\)\s*$"

I believe there are other places that could be improved in the same way. Please address them as well if possible.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

dkearns

unread,

9:36 AM (2 hours ago) 9:36 AM

to vim/vim, Subscribed

dkearns left a comment (vim/vim#20191)

Thanks.

I haven't had a close look but I think the syntax file changes are going to be easier to merge after #19331, rather than vice versa. I'll try and finish that this week.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.