[vim/vim] Added Python runtime syntax soft keywords (PR #10621)

Diego Frias

unread,

Jun 25, 2022, 11:13:26 PM6/25/22

to vim/vim, Subscribed

Problem

In Python's current syntax highlighting, the words match and case are always highlighted, which could give the wrong idea about their usage. match and case are only given special meaning when used in a specific context, so it would be misleading to highlight them under the same conditions of if and else.

Fix

Added match and case as soft keywords, as described here. Modeled after Python 3.10's own IDLE behavior when dealing with soft keywords.

I contacted the maintainer of the file about this pull request.

You can view, comment on, or merge this pull request online at:

https://github.com/vim/vim/pull/10621

Commit Summary

ad1b365 Added Python runtime syntax soft keywords

File Changes

(1 file)

M runtime/syntax/python.vim (11)

Patch Links:

—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.

Doug Kearns

unread,

Jun 26, 2022, 12:22:19 AM6/26/22

to vim/vim, Subscribed

@dkearns commented on this pull request.

In runtime/syntax/python.vim:

>  syn keyword pythonRepeat	for while
 syn keyword pythonOperator	and in is not or
 syn keyword pythonException	except finally raise try
 syn keyword pythonInclude	from import
 syn keyword pythonAsync		async await
 
+" Soft keywords
+" These keywords do not mean anything unless used in the right context
+" See https://peps.python.org/pep-0634/#the-match-statement for more on this.
+" In Python 3.10's IDLE for example, 'match' and 'case' are only highlighted
+" when in the right context
+syn match   pythonConditional   "\v^\s*case(.*:$)@="
+syn match   pythonConditional   "\v^\s*match(.*:$)@="

It looks like you need to allow trailing whitespace after the colons.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Diego Frias

unread,

Jun 26, 2022, 12:35:06 AM6/26/22

to vim/vim, Push

@dzfrias pushed 1 commit.

7eaa4d1 Python soft keywords work with trailing whitespace

—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.

Diego Frias

unread,

Jun 26, 2022, 12:36:47 AM6/26/22

to vim/vim, Subscribed

@dzfrias commented on this pull request.

In runtime/syntax/python.vim:

>  syn keyword pythonRepeat	for while
 syn keyword pythonOperator	and in is not or
 syn keyword pythonException	except finally raise try
 syn keyword pythonInclude	from import
 syn keyword pythonAsync		async await
 
+" Soft keywords
+" These keywords do not mean anything unless used in the right context
+" See https://peps.python.org/pep-0634/#the-match-statement for more on this.
+" In Python 3.10's IDLE for example, 'match' and 'case' are only highlighted
+" when in the right context
+syn match   pythonConditional   "\v^\s*case(.*:$)@="
+syn match   pythonConditional   "\v^\s*match(.*:$)@="

I think this should be fixed now. Thanks for letting me know

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Jun 26, 2022, 7:02:07 AM6/26/22

to vim/vim, Subscribed

@zvezdan

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Zvezdan Petkovic

unread,

Jun 27, 2022, 2:47:52 PM6/27/22

to vim/vim, Subscribed

@zvezdan requested changes on this pull request.

Thank you for noticing this issue and working on the fix.
I asked for some changes.
If they are not clear or you don't feel like working out all the complexities, feel free to let me know and I'll add the changes to a later update of the syntax file.

In runtime/syntax/python.vim:

> +syn match   pythonConditional   "\v^\s*case(.*:\s*$)@="
+syn match   pythonConditional   "\v^\s*match(.*:\s*$)@="

A few issues with these regular expressions:

Both case and match must be followed by a \s before any other content -- match subject or case pattern(s). These regular expressions are not enforcing this space separation and would accept matchX or caseY. There needs to be a \s as the first part of the content inside the parentheses for zero-width look-ahead match (@=).
According to grammar, the match subject: must be followed by a newline, indented block, and dedent, which allows for the check of $ at the end of the regex. The case pattern: is followed by a block according to grammar. This means that besides "newline, indented block, dedent" it can be followed by a simple statement. So a check for $ is not necessarily correct in that case. It can be a \s*$, but it could also be other characters that could make a simple statement between the colon and the end of the line.
The patterns in this file never use \v (very magic) and I'd keep the prevailing style. The non-capturing parentheses could be used \%( ... \)@= and are the only part of the regex that requires a slightly more complex writing without \v which doesn't seem too bad.
The ^\s* makes the space in front of match or case the part of this match, while only the keywords should be matched. See some examples in the other regular expressions in this file that set \zs for the start of the match after finding the pre-requisite part in front of the match.

In runtime/syntax/python.vim:

> +" These keywords do not mean anything unless used in the right context
+" See https://peps.python.org/pep-0634/#the-match-statement for more on this.

I would refer to the Python documentation section on soft keywords instead: https://docs.python.org/3/reference/lexical_analysis.html#soft-keywords

That should be enough without any further explanation except the heading Soft keywords.

In runtime/syntax/python.vim:

> +" In Python 3.10's IDLE for example, 'match' and 'case' are only highlighted
+" when in the right context

This is irrelevant. Grammar is the relevant reason why we need this change, not what IDLE does. I would remove this part of the comment.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Diego Frias

unread,

Jun 28, 2022, 10:41:35 AM6/28/22

to vim/vim, Push

@dzfrias pushed 1 commit.

80d7312 Improve match and case highlighting

—
View it on GitHub or unsubscribe.

You are receiving this because you are subscribed to this thread.

Diego Frias

unread,

Jun 28, 2022, 10:44:06 AM6/28/22

to vim/vim, Subscribed

Thanks for reviewing this! I added the changes requested, tell me if I did something wrong. I also added the ability to make inline comments after a match statement.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Zvezdan Petkovic

unread,

Jul 27, 2022, 10:20:23 PM7/27/22

to vim/vim, Subscribed

@zvezdan commented on this pull request.

In runtime/syntax/python.vim:

>  syn keyword pythonRepeat	for while

 syn keyword pythonOperator	and in is not or

 syn keyword pythonException	except finally raise try

 syn keyword pythonInclude	from import

 syn keyword pythonAsync		async await

 

+" Soft keywords

+" These keywords do not mean anything unless used in the right context

+" See https://docs.python.org/3/reference/lexical_analysis.html#soft-keywords 

+" for more on this.

+syn match   pythonConditional   "^\s*\zscase\%(\s.*:.*$\)\@="

It's probably not frequent, but one could write case 4 which conforms to the grammar as much as case 4 -- multiple spaces after case vs. a single space. So, I think we should have \s\+ after case and before .*:.

In runtime/syntax/python.vim:

>  syn keyword pythonRepeat	for while

 syn keyword pythonOperator	and in is not or

 syn keyword pythonException	except finally raise try

 syn keyword pythonInclude	from import

 syn keyword pythonAsync		async await

 

+" Soft keywords

+" These keywords do not mean anything unless used in the right context

+" See https://docs.python.org/3/reference/lexical_analysis.html#soft-keywords 

+" for more on this.

+syn match   pythonConditional   "^\s*\zscase\%(\s.*:.*$\)\@="

+syn match   pythonConditional   "^\s*\zsmatch\%(\s.*:\s*\%(#.*\)\?$\)\@="

Same comment about \s\+ after match and before .*:.
Additionally, the regular expressions in this file use \= instead of \? consistently. Perhaps we can get that replaced too, together with the multi-space fix?

Again, thank you, and sorry for the long delay for this second review. I was unavailable, but next review should be quick. This is looking practically ready. In fact, if you're tired of changing it, we might commit as is and I can change in the subsequent commit. If you don't mind making these two small changes, even better. 🙂

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

Diego Frias

unread,

Jul 27, 2022, 10:59:15 PM7/27/22

to vim/vim, Push

@dzfrias pushed 1 commit.

4684b7f Improve python match and case highlighting

—
View it on GitHub or unsubscribe.

You are receiving this because you are subscribed to this thread.

Diego Frias

unread,

Jul 27, 2022, 11:01:29 PM7/27/22

to vim/vim, Subscribed

@dzfrias commented on this pull request.

In runtime/syntax/python.vim:

>  syn keyword pythonRepeat	for while
 syn keyword pythonOperator	and in is not or
 syn keyword pythonException	except finally raise try
 syn keyword pythonInclude	from import
 syn keyword pythonAsync		async await
 
+" Soft keywords
+" These keywords do not mean anything unless used in the right context
+" See https://docs.python.org/3/reference/lexical_analysis.html#soft-keywords

 
+" for more on this.
+syn match   pythonConditional   "^\s*\zscase\%(\s.*:.*$\)\@="
+syn match   pythonConditional   "^\s*\zsmatch\%(\s.*:\s*\%(#.*\)\?$\)\@="

I added those changes. No problem at all, hopefully they look okay now

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Diego Frias

unread,

Jul 27, 2022, 11:02:08 PM7/27/22

to vim/vim, Subscribed

Do these look good now @zvezdan? Let me know if I need to do more or messed something up.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Zvezdan Petkovic

unread,

Jul 27, 2022, 11:27:00 PM7/27/22

to vim/vim, Subscribed

@zvezdan approved this pull request.

Looks good!
Thanks!

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Zvezdan Petkovic

unread,

Jul 27, 2022, 11:29:50 PM7/27/22

to vim/vim, Subscribed

Do these look good now @zvezdan? Let me know if I need to do more or messed something up.

Yes. I just tested them also and the highlighting looks correct.
Thanks for changing this.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Jul 28, 2022, 6:22:48 AM7/28/22

to vim/vim, Subscribed

I'll include it, thanks

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Jul 28, 2022, 6:22:51 AM7/28/22

to vim/vim, Subscribed

Closed #10621.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Reply all

Reply to author

Forward