[vim/vim] Vim9: "g" wrongly parsed as global namespace in "g:pat:yank" (#6593)

[lacygoill]

Describe the bug

In g:pat:yank, g is wrongly parsed as the global namespace, and pat as the name of a global variable.

To Reproduce

Run this shell command:

vim -Nu NONE -i NONE -S <(cat <<'EOF'
    vim9script
    let lines =<< trim END
        aaa
        pat
        bbb
        pat
    END
    setline(1, lines)
    g:pat:yank A
EOF
)

E121 is raised:

E121: Undefined variable: g:pat

Expected behavior

No error is raised, and all the lines matching the pattern pat are yanked in the a register.

Environment

• Vim version: 8.2 Included patches: 1-1333
• OS: Ubuntu 16.04.6 LTS
• Terminal: XTerm(358)

Additional context

According to :h :g, it is allowed to use a delimiter other than a slash to surround the pattern of a global command:
https://github.com/vim/vim/blob/b53da7918c643ef4de1256c37bc8b92413e6dcec/runtime/doc/repeat.txt#L62-L63

This is confirmed by the fact that the same code works as expected in a legacy Vim script:

let lines =<< trim END
    aaa
    pat
    bbb
    pat
END
call setline(1, lines)
g:pat:yank A

---

As a workaround, a space can be added in-between g and ::

g :pat:yank A
 ^

Or we can use the full name of the global command:

global:pat:yank A
^----^

---

FWIW, I wouldn't mind if it was decided to enforce writing names of Ex commands and options in their full form.

It would force us to write more readable code, and avoid this kind of issues.

Here is another example of pitfalls which was already mentioned in #6399, and would be fixed for the most part if we had to write names of Ex commands in their full form:

vim9script
v = 123

E749: empty buffer

The error is expected because the variable was not declared. What will be less expected for users, and I suspect what will cause many issue reports, is:

• the fact that the error message is unreadable
  ("I did not ask to do anything in the buffer, so why does Vim complain about the latter?")

• possible side-effects of the command matching the variable name

Here is a summary of errors for single-letter characters:

┌───┬───────────────────────────────────────────┐
│ b │ E94: No matching buffer for = 123         │
├───┼───────────────────────────────────────────┤
│ e │ edit file '= 123'                         │
├───┼───────────────────────────────────────────┤
│ f │ E95: Buffer with this name already exists │
├───┼───────────────────────────────────────────┤
│ g │ Pattern not found:  123                   │
├───┼───────────────────────────────────────────┤
│ h │ E149: Sorry, no help for = 123            │
├───┼───────────────────────────────────────────┤
│ m │ E16: Invalid range                        │
├───┼───────────────────────────────────────────┤
│ n │ populate arglist with '=' and '123'       │
├───┼───────────────────────────────────────────┤
│ o │ edit file '= 123'                         │
├───┼───────────────────────────────────────────┤
│ r │ E484: Can't open file = 123               │
├───┼───────────────────────────────────────────┤
│ s │ E486: Pattern not found:  123             │
├───┼───────────────────────────────────────────┤
│ t │ E16: Invalid range                        │
├───┼───────────────────────────────────────────┤
│ v │ E749: empty buffer                        │
├───┼───────────────────────────────────────────┤
│ w │ E139: File is loaded in another buffer    │
├───┼───────────────────────────────────────────┤
│ z │ E144: non-numeric argument to :z          │
└───┴───────────────────────────────────────────┘

Incidentally, the google Vim script style guide already recommends writing names of settings in their full form:

Settings
Prefer long names.

I'm also all for it if it helps the developers fix bugs and implement new features, like a parser for Vim9 script.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

[lacygoill]

As a workaround, a space can be added in-between g and ::

g :pat:yank A
 ^

Or we can use the full name of the global command:

global:pat:yank A
^----^

Or we can prefix the command with a colon:

:g:pat:yank A
^

[Bram Moolenaar]

As we already require prefixing a range with a colon, we should do the same for any ambiguous command.
I don't really want to require it always, as using ":edit file" looks strange, and there are only a few cases
where there is confusion. For the original example "g:pat:yank A" you get an error that is a good hint.
I looked into recognizing it, but since "pat" can be any pattern, it's not easy to explain what would still work.
Better assume everything starting with "g:" is a function or variable. Using "g:pattern:cmd" is uncommon enough.

[Bram Moolenaar]

Closed #6593.

[lacygoill]

There is another issue with the - delimiter.

vim9script
g->a-p

Expected:

Pattern not found: >a

Actual:

E121: Undefined variable: g

There is an ambiguity between g the global command, and g the variable name. If we remove > at the start of the pattern:

vim9script
g-a-p

Pattern not found: a

Then Vim correctly parses g as the global command.

I think the same issue applies to any command expecting a pattern as argument. That is, if we choose - as a delimiter, then we better make sure that the pattern doesn't start with >, so that Vim doesn't parse the combination as a method call.

I'm trying to write a syntax plugin for Vim9. I could somewhat handle this case with an extra rule, but I think it's not worth it. Same thing for the colon delimiter; I would need 2 extra rules to somewhat handle it.

That's too much extra code, so I just don't recognize : nor - as delimiters. Most of the time, people use / anyway. For the few cases where / is not desired, there are more than enough alternatives in the ascii table. ;, ,, a backtick ...

FWIW, instead of documenting all these corner cases in the help, I would just disallow : and - as delimiters. This would be less confusing; no exceptions to exceptions...

---

There might also be an issue with :h pattern-delimiter. The latter says that we can't use a double quote as a delimiter around a pattern passed as an argument to a command such as :substitute or :global:

pattern-delimiter E146
Instead of the '/' which surrounds the pattern and replacement string, you
can use any other single-byte character, but not an alphanumeric character,
'', '"' or '|'. This is useful if you want to include a '/' in the search
pattern or replacement string.

In Vim9, that's still true:

vim9script
['aba bab']->repeat(3)->setline(1)
sil! s/nowhere//
:% s"b"B"g

E486: Pattern not found: nowhere

But should it? " is no longer a comment leader in Vim9.

Also, since # is the comment leader in Vim9, should it be disallowed in Vim9? Right now, it works:

vim9script
['aba bab']->repeat(3)->setline(1)
sil! s/nowhere//
:% s#b#B#g
:% p

aBa BaB
aBa BaB
aBa BaB

[Bram Moolenaar]

> There is another issue with the `-` delimiter.

> ```vim

> vim9script
> g->a-p
> ```
> Expected:
>
> Pattern not found: >a
>
> Actual:
>
> E121: Undefined variable: g
>
> There is an ambiguity between `g` the global command, and `g` the
> variable name. If we remove `>` at the start of the pattern:

> ```vim

> vim9script
> g-a-p
> ```
> Pattern not found: a
>
> Then Vim correctly parses `g` as the global command.

Yeah, there is ambiguety between using the "g" command and a "g"
variable. In this case the "->" is recognized as the method operator,
and that wins. We could also check if the "g" variable exists, but that
probably makes it more complicated.

> I think the same issue applies to any command expecting a pattern as
> argument. That is, if we choose `-` as a delimiter, then we better
> make sure that the pattern doesn't start with `>`, so that Vim doesn't
> parse the combination as a method call.

Using "-" as delimiter and ">" as the first character in the pattern is
very, very unlikely to happen. If it does, then it's easy to use
another delimiter.

> I'm trying to write a syntax plugin for Vim9. I could somewhat handle this case with an extra rule, but I think it's not worth it. Same thing for the colon delimiter; I would need 2 extra rules to *somewhat* handle it.

>
> That's too much extra code, so [I just don't recognize `:` nor `-` as

> delimiters](https://github.com/lacygoill/vim9-syntax/blob/260e92defec1b6c9a3bc42aca2708a57fd995f15/syntax/vim.vim#L927-L995).

> Most of the time, people use `/` anyway. For the few cases where `/`
> is not desired, there are more than enough alternatives in the ascii

> table. `;`, `,`, a backtick ...

For highlighting it's probably OK to not recognize ":" and "-" as
delimiter. You defenitely don't want to check that "g" was declared
before.

> FWIW, instead of documenting all these corner cases in the help, I
> would just disallow `:` and `-` as delimiters. This would be less
> confusing; no exceptions to exceptions...

The :g command is very old, I don't think we want to change it with what
appears to be arbitrary restrictions.

> There might also be an issue with `:h pattern-delimiter`. The latter

> says that we can't use a double quote as a delimiter around a pattern

> passed as an argument to a command such as `:substitute` or `:global`:
>
> > *pattern-delimiter* *E146*

> > Instead of the '/' which surrounds the pattern and replacement string, you
> > can use any other single-byte character, but not an alphanumeric character,
> > '\', '"' or '|'. This is useful if you want to include a '/' in the search
> > pattern or replacement string.
>
> In Vim9, that's still true:

> ```vim

> vim9script
> ['aba bab']->repeat(3)->setline(1)
> sil! s/nowhere//
> :% s"b"B"g
> ```
> E486: Pattern not found: nowhere
>
> But should it? `"` is no longer a comment leader in Vim9.
>
> Also, since `#` is the comment leader in Vim9, should it be disallowed in Vim9? Right now, it works:

> ```vim

> vim9script
> ['aba bab']->repeat(3)->setline(1)
> sil! s/nowhere//
> :% s#b#B#g
> :% p
> ```
> aBa BaB
> aBa BaB
> aBa BaB

The # only starts a comment when there is a space before it.

Generally, I think you are running into these corner cases because of
what you are doing. A normal user probably never encounters this, or
can easily use another delimiter.

--
How To Keep A Healthy Level Of Insanity:
18. When leaving the zoo, start running towards the parking lot,
yelling "run for your lives, they're loose!!"

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[lacygoill]

The :g command is very old, I don't think we want to change it with what
appears to be arbitrary restrictions.

Actually, I was suggesting to change :h pattern-delimiter, so that in addition to \, ", |, which are already currently disallowed, we also disallow : and -. To be consistent, because I find it confusing that some delimiters work in some cases but not all. It's easier to just disallow them all the time for all commands. Even if they are not really disallowed syntactically, at least discourage their usage in this help tag. Just my opinion.

The # only starts a comment when there is a space before it.

Ah so that's where the difference come from. In legacy, there is no such requirement for ", which is why – I guess – it's disallowed in Vim9. Although, # still works even when preceded by a whitespace:

vim9script

['aba bab']->repeat(3)->setline(1)

sil! s/nowhere/

/

:% s #b#B#g

:% p

I'll still not allow it because I'm not sure it's always reliable.
For example, with :filter, in legacy it works:

filter #pat# ls

But not in Vim9:

vim9script

filter #pat# ls

E476: Invalid command: filter #pat# ls

And yet, still in Vim9, # does work for :g:

vim9script

g #pat# eval 0

Pattern not found: pat

This is too confusing, so I disallow #, just like " was disallowed in legacy.