[vim/vim] iskeyword documentation lacks clarity on unicode (Issue #18688)

[Maxim Kim]

habamax created an issue (vim/vim#18688)

Steps to reproduce

iskeyword doc section mentions:

	Keywords are used in searching and recognizing with many commands:
	"w", "*", "[i", etc.  It is also used for "\k" in a |pattern|.  See
	'isfname' for a description of the format of this option.  For '@'
	characters above 255 check the "word" character class (any character
	that is not white space or punctuation).

Mentioning "word" character class doesn't make sense here (if we look it up in the help):

\w	word character:			[0-9A-Za-z_]	*/\w*

So we assume continuous (any character that is not white space or punctuation) is a word.

With this, my understanding is that w or K should work with a keyword that has unicode chars as well as \k character class in regexes.

This is the test unicode string:

hello₇₇₇worldand🦢🦢🦢🦢swans

image.png (view on web)

1. If I search for the /\k\+ the whole line is found as a keyword -- that is expected
2. however if I press w it would not go till the end of the word -- that is not expected
3. the same if I press K keywordprg will only try to open part of the keyword (hello or ₇₇₇ or 🦢🦢🦢🦢 etc.

The source https://www.reddit.com/r/vim/comments/1op95ga/unicode_characters_in_iskeyword/

https://asciinema.org/a/Z8rNAwQ85fpplAVdkF2n1M3JO

Expected behaviour

Either clarified doc section or fix inconsisteny in handling iskeyword with w, e, K and other related commands.

Version of Vim

9.1.1898

Environment

debian13, bash

Logs and stack traces

—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/issues/18688@github.com>

[Christian Brabandt]

chrisbra left a comment (vim/vim#18688)

So what is happening here:

hello₇₇₇worldand🦢🦢🦢🦢swans

I guess we all agree, hat hello, world , and swans are all word characters. So the issue is with:

'🦢' U+1F9A2 Dec:129442 SWAN
'₇' U+2087 Dec:8327 SUBSCRIPT SEVEN

According to the Unicode general category, the first one is part of the emoji table and so is considered a word character. The Subscript is also a considered a word character, but internally Vim builds it's own character class subscript and while considered a word, w stops when traversing over different classes of words. For the same reason w stops when moving over the swan emoji.

Let me update the help a bit more.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/issues/18688/3873755058@github.com>

[Christian Brabandt]

Closed #18688 as completed via 9e456e5.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/issue/18688/issue_event/22654320367@github.com>