[vim/vim] Add \%f) and \%t) regexp atoms matching to a closing delimiter (PR #21244)
rhz
Adds pattern atoms that match forward to the delimiter closing the nesting level the match is on, counting nested pairs on the way:
\%f) \%f] \%f} \%f> include the closing delimiter ("find", like f)
\%t) \%t] \%t} \%t> stop just before it ("till", like t)
\%) \%] \%} \%> short for the \%f form
Each has a \_ form (\_%), \_%f), \_%t) ...) that keeps searching on the following lines, so /{\_%f} matches a whole brace block however many lines it covers.
Counting nesting depth is not something a regular expression can express, so this is not reachable by combining existing atoms. The nearest tools are % and searchpair(), which find a matching delimiter without a pattern.
Implemented in both the backtracking and the NFA engine, with documentation in pattern.txt and tests in test_regexp_latin.vim and test_regexp_utf8.vim covering both engines, multi-line matching, multibyte text and a double-byte encoding.
Behaviour change to note: \%> is the new atom only when no digit, . or ' follows, so \%>23l, \%>.c, \%>'m etc. keep their meaning. As a consequence a previously invalid pattern such as \%>v now compiles and simply finds nothing (E486 instead of E71); Test_column_success_failure is updated accordingly.
AI disclosure: this change was developed with the help of Claude (Anthropic). I reviewed it, built it and ran the test suite before submitting.
Tests run locally: test_regexp_latin, test_regexp_utf8, test_search, test_substitute, test_syntax, test_help, test_gn and codestyle (the only codestyle failures are in sign.c and sound.c, which this change does not touch).
🤖 Generated with Claude Code
https://claude.ai/code/session_01V3mzfghg6icyf5RgQAW6oZ
You can view, comment on, or merge this pull request online at:
  https://github.com/vim/vim/pull/21244
Commit Summary
- 1c63ea1 feat(regexp): add \%f) \%t) atoms matching to a closing delimiter
File Changes
(6Â files)
- M runtime/doc/pattern.txt (55)
- M runtime/doc/tags (9)
- M src/regexp_bt.c (219)
- M src/regexp_nfa.c (381)
- M src/testdir/test_regexp_latin.vim (238)
- M src/testdir/test_regexp_utf8.vim (44)
Patch Links:
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
Christian Brabandt
chrisbra left a comment (vim/vim#21244)Thanks, but those are a lot of special atoms for dubious gains I think?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
rhz
arbace left a comment (vim/vim#21244)Thank you for taking a look, and for the honest reaction.
I understand the concern about the number of spellings. I would argue it is really one atom, "match to the delimiter closing this level", with the same three variations Vim already has elsewhere: which of the four bracket pairs (as % does), include the delimiter or stop before it (as f and t do), and stay on the line or cross line breaks (as \_ does for . and []). The table in the docs looks long only because it lists every combination.
What made me write it is that there is no replacement. Counting nested pairs is beyond what a regular expression can express, so /(\%f) with nested parentheses cannot be built from existing atoms. % and searchpair() find a matching delimiter, but they cannot be used inside a pattern: not in :s, :g, matchstr(), matchadd(), :syntax, or in a search with other text around the block. I use \%) daily for exactly those cases, to the point where I no longer manage well with an unpatched Vim.
On the cost side: nothing changes for patterns that do not use it, both engines implement it natively so no fallback to the backtracking engine is needed, and the only behaviour change is that a previously invalid pattern like \%>v now compiles instead of giving E71. The change comes with tests for both engines, multi-line matching, multibyte text and a double-byte encoding.
That said, it is of course your call, and I will fully understand if you feel it does not belong in Vim. In that case I will simply keep carrying it in my fork. Thanks again for your time.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
h_east
h-east left a comment (vim/vim#21244)I am against adding this. Two reasons, apart from the number of spellings.
It does not fit the grammar of Vim patterns. Every existing atom matches
one character, a position (zero width) or an explicitly listed sequence. No
atom scans ahead and consumes text on its own; consuming text is always the
job of a multi (*, \+, \{n,m}). That is why (\%) matching the whole
(a(b)c) without any .* looks wrong: the atom carries a hidden loop. The
two variable-length atoms Vim has are different in kind: ~ matches a fixed
string of variable length, and \%[] matches a sequence that is spelled out
in the pattern. Engines that support balanced delimiters do it as a grammar
extension (recursion in PCRE, balancing groups in .NET), not as a special
atom, and Vim has nothing of that kind to build on.
Three of the four spellings collide with existing \% items. \%( opens
a non-capturing group, so \%) reads as its closing counterpart while the
real one stays \). \%[ starts an optional sequence, so \%] reads as
its end. \%< and \%> are position atoms, and with this change the meaning
of \%> depends on the character after it; a pattern such as \%>v that
used to be rejected with E71 now compiles and matches nothing.
Matching a balanced block is a real need, but Vim covers it outside of
patterns: %, searchpair(), the text objects (i(, a{, ...) and syntax
regions with "contains". Putting it inside the pattern syntax costs a
non-regular construct in regexp_bt.c and regexp_nfa.c, and I do not think the
gain justifies that.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
rhz
arbace left a comment (vim/vim#21244)Thank you for the detailed review, that is a fair description of the trade-off.
I take the point about the grammar: this atom does consume text on its own, and Vim has nothing comparable to build on. That is a design objection I cannot argue away; I can only say that in daily use I have found the construct valuable enough to be worth that irregularity, and that the tools outside patterns do not help in :s, :g, matchstr() or :syntax. On the \% spellings, I would only note that \%( and \%[ never had a \% closing counterpart, and that \%>v was an error before, so no working pattern changes meaning.
But this is a judgement call about what belongs in Vim, and that is for the maintainers to make. If the decision is not to take it, no hard feelings at all: I will keep the change in my fork. Thanks to both of you for your time.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
h_east
h-east left a comment (vim/vim#21244)and that %>v was an error before, so no working pattern changes meaning.
With this PR a pattern that fails with E71 on master compiles and matches
nothing. That is a change in behavior, and the PR has to adjust
Test_column_success_failure for it. In practice a typo such as \%>v for
\%>5v is no longer reported; this PR turns it into a search that silently
finds nothing.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
Christian Brabandt
chrisbra left a comment (vim/vim#21244)I agree, in its current form it doesn't fit into Vim since it consumes the complete range and it is not clear how it would handle following quantifier. I wonder if this would be cleaner as a pair of zero-width assertions instead, e.g.:
\%s[ ... \%e]
where \%s[ marks the start delimiter and \%e] asserts the corresponding closing delimiter. Nesting would be tracked between the \%s assertions in order. So this /^\%s(foo.*\%e) would match a line like this: (foo (bar)) or (foobar) but not (foobaz
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.