[vim/vim] Expanding special characters anywhere in a command string (#4514)

[Yegappan Lakshmanan]

The expand() function can be used to expand special characters. If the special
characters appears anywhere in the command string, then this cannot be used.

Vim supports expanding special characters anywhere in a command string
for internal commands. For example, the 'makeprg' option accepts a command
string where special characters can occur anywhere in the string.
But a Vim plugin cannot support this. So plugins like dispatch.vim use a complicated
function to expand the special characters (look at the dispatch#expand() function in
https://github.com/tpope/vim-dispatch/blob/master/autoload/dispatch.vim).

This patch adds a expandcmd() function that can be used to expand all the
special characters in a command string.

---

You can view, comment on, or merge this pull request online at:

  https://github.com/vim/vim/pull/4514

Commit Summary

• Add the expandcmd() function

File Changes

• M runtime/doc/eval.txt (9)
• M src/evalfunc.c (29)
• M src/testdir/test_expand.vim (31)

Patch Links:

• https://github.com/vim/vim/pull/4514.patch
• https://github.com/vim/vim/pull/4514.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub

[Codecov]

Codecov Report

Merging #4514 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@

##           master    #4514      +/-   ##

==========================================

+ Coverage   80.63%   80.64%   +<.01%     

==========================================

  Files         110      110              

  Lines      143701   143713      +12     

==========================================

+ Hits       115874   115898      +24     

+ Misses      27827    27815      -12

Impacted Files	Coverage Δ
src/evalfunc.c	89.31% <100%> (+0.02%)	⬆️
src/if_xcmdsrv.c	83.84% <0%> (-0.18%)	⬇️
src/channel.c	83.37% <0%> (-0.08%)	⬇️
src/message.c	76.89% <0%> (+0.04%)	⬆️
src/window.c	87.28% <0%> (+0.09%)	⬆️
src/gui.c	58.21% <0%> (+0.1%)	⬆️
src/gui_gtk_x11.c	49.55% <0%> (+0.44%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 260addf...4136d34. Read the comment docs.

—
You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or mute the thread.

[Yegappan Lakshmanan]

@yegappan pushed 1 commit.

• 7492d68 Update the test and doc.

—
You are receiving this because you are subscribed to this thread.

View it on GitHub

[Bram Moolenaar]

Closed #4514 via 80dad48.

[Bram Moolenaar]

Yegappan wrote:

> The expand() function can be used to expand special characters. If the
> special characters appears anywhere in the command string, then this
> cannot be used.
>
> Vim supports expanding special characters anywhere in a command string
> for internal commands. For example, the 'makeprg' option accepts a command
> string where special characters can occur anywhere in the string.
> But a Vim plugin cannot support this. So plugins like dispatch.vim use a complicated
> function to expand the special characters (look at the dispatch#expand() function in
> https://github.com/tpope/vim-dispatch/blob/master/autoload/dispatch.vim).
>
> This patch adds a expandcmd() function that can be used to expand all the
> special characters in a command string.

Thanks. I updated the docs to mention that the expansion happens like
for the ":edit" command. Other commands, such as ":next" work slightly
differently.

I also reported the error, when there is one. There is still the case
it returns with an error, but without an error message. Perhaps we
should also fail then with some generic message?

--
hundred-and-one symptoms of being an internet addict:
134. You consider bandwidth to be more important than carats.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Yegappan Lakshmanan]

Hi Bram,

On Sun, Jun 9, 2019 at 9:05 AM Bram Moolenaar <vim-dev...@256bit.org> wrote:
>
> Yegappan wrote:
>
> > The expand() function can be used to expand special characters. If the
> > special characters appears anywhere in the command string, then this
> > cannot be used.
> >
> > Vim supports expanding special characters anywhere in a command string
> > for internal commands. For example, the 'makeprg' option accepts a command
> > string where special characters can occur anywhere in the string.
> > But a Vim plugin cannot support this. So plugins like dispatch.vim use a complicated
> > function to expand the special characters (look at the dispatch#expand() function in
> > https://github.com/tpope/vim-dispatch/blob/master/autoload/dispatch.vim).
> >
> > This patch adds a expandcmd() function that can be used to expand all the
> > special characters in a command string.
>
> Thanks. I updated the docs to mention that the expansion happens like
> for the ":edit" command. Other commands, such as ":next" work slightly
> differently.
>

Thanks for merging the patch. Hopefully this is sufficient for plugins
to expand options like 'makeprg', 'grepprg', 'equalprg', 'formatprg',
'keywordprg', etc.

>
> I also reported the error, when there is one. There is still the case
> it returns with an error, but without an error message. Perhaps we
> should also fail then with some generic message?
>

Are you referring to the case where expand_filename() returns an
empty error message string?

Regards,
Yegappan

[vim-dev ML]

Hi Bram,

On Sun, Jun 9, 2019 at 9:05 AM Bram Moolenaar <vim-dev...@256bit.org> wrote:
>
> Yegappan wrote:
>

> > The expand() function can be used to expand special characters. If the
> > special characters appears anywhere in the command string, then this
> > cannot be used.
> >
> > Vim supports expanding special characters anywhere in a command string
> > for internal commands. For example, the 'makeprg' option accepts a command
> > string where special characters can occur anywhere in the string.
> > But a Vim plugin cannot support this. So plugins like dispatch.vim use a complicated
> > function to expand the special characters (look at the dispatch#expand() function in
> > https://github.com/tpope/vim-dispatch/blob/master/autoload/dispatch.vim).
> >
> > This patch adds a expandcmd() function that can be used to expand all the
> > special characters in a command string.
>

> Thanks. I updated the docs to mention that the expansion happens like
> for the ":edit" command. Other commands, such as ":next" work slightly
> differently.
>

Thanks for merging the patch. Hopefully this is sufficient for plugins
to expand options like 'makeprg', 'grepprg', 'equalprg', 'formatprg',
'keywordprg', etc.

>
> I also reported the error, when there is one. There is still the case
> it returns with an error, but without an error message. Perhaps we
> should also fail then with some generic message?
>

Are you referring to the case where expand_filename() returns an
empty error message string?

Regards,
Yegappan

[Bram Moolenaar]

Yes.

--
hundred-and-one symptoms of being an internet addict:

139. You down your lunch in five minutes, at your desk, so you can
spend the rest of the hour surfing the Net.

[vim-dev ML]

Yegappan wrote:

> > > The expand() function can be used to expand special characters. If the
> > > special characters appears anywhere in the command string, then this
> > > cannot be used.
> > >
> > > Vim supports expanding special characters anywhere in a command string
> > > for internal commands. For example, the 'makeprg' option accepts a command
> > > string where special characters can occur anywhere in the string.
> > > But a Vim plugin cannot support this. So plugins like dispatch.vim use a complicated
> > > function to expand the special characters (look at the dispatch#expand() function in
> > > https://github.com/tpope/vim-dispatch/blob/master/autoload/dispatch.vim).
> > >
> > > This patch adds a expandcmd() function that can be used to expand all the
> > > special characters in a command string.
> >

> > Thanks. I updated the docs to mention that the expansion happens like
> > for the ":edit" command. Other commands, such as ":next" work slightly
> > differently.
> >
>
> Thanks for merging the patch. Hopefully this is sufficient for plugins
> to expand options like 'makeprg', 'grepprg', 'equalprg', 'formatprg',
> 'keywordprg', etc.
>
> >
> > I also reported the error, when there is one. There is still the case
> > it returns with an error, but without an error message. Perhaps we
> > should also fail then with some generic message?
> >
>
> Are you referring to the case where expand_filename() returns an
> empty error message string?

Yes.

--
hundred-and-one symptoms of being an internet addict:
139. You down your lunch in five minutes, at your desk, so you can
spend the rest of the hour surfing the Net.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Yegappan Lakshmanan]

Hi,

On Sun, Jun 9, 2019 at 9:15 AM Yegappan Lakshmanan <yega...@gmail.com> wrote:
>
> >
> > Yegappan wrote:
> >
> > > The expand() function can be used to expand special characters. If the
> > > special characters appears anywhere in the command string, then this
> > > cannot be used.
> > >
> > > Vim supports expanding special characters anywhere in a command string
> > > for internal commands. For example, the 'makeprg' option accepts a command
> > > string where special characters can occur anywhere in the string.
> > > But a Vim plugin cannot support this. So plugins like dispatch.vim use a complicated
> > > function to expand the special characters (look at the dispatch#expand() function in
> > > https://github.com/tpope/vim-dispatch/blob/master/autoload/dispatch.vim).
> > >
> > > This patch adds a expandcmd() function that can be used to expand all the
> > > special characters in a command string.
> >
> > Thanks. I updated the docs to mention that the expansion happens like
> > for the ":edit" command. Other commands, such as ":next" work slightly
> > differently.
> >
>
> Thanks for merging the patch. Hopefully this is sufficient for plugins
> to expand options like 'makeprg', 'grepprg', 'equalprg', 'formatprg',
> 'keywordprg', etc.
>

This will be useful for plugins which parse the above options and
start a job asynchronously using job_start(). To do this, these
plugins need to expand the special items in the above options
and then pass the resulting strings as arguments to job_start().

- Yegappan

[vim-dev ML]

Hi,

On Sun, Jun 9, 2019 at 9:15 AM Yegappan Lakshmanan <yega...@gmail.com> wrote:
>
> >
> > Yegappan wrote:
> >

> > > The expand() function can be used to expand special characters. If the
> > > special characters appears anywhere in the command string, then this
> > > cannot be used.
> > >
> > > Vim supports expanding special characters anywhere in a command string
> > > for internal commands. For example, the 'makeprg' option accepts a command
> > > string where special characters can occur anywhere in the string.
> > > But a Vim plugin cannot support this. So plugins like dispatch.vim use a complicated
> > > function to expand the special characters (look at the dispatch#expand() function in
> > > https://github.com/tpope/vim-dispatch/blob/master/autoload/dispatch.vim).
> > >
> > > This patch adds a expandcmd() function that can be used to expand all the
> > > special characters in a command string.
> >

> > Thanks. I updated the docs to mention that the expansion happens like
> > for the ":edit" command. Other commands, such as ":next" work slightly
> > differently.
> >
>
> Thanks for merging the patch. Hopefully this is sufficient for plugins
> to expand options like 'makeprg', 'grepprg', 'equalprg', 'formatprg',
> 'keywordprg', etc.
>

This will be useful for plugins which parse the above options and
start a job asynchronously using job_start(). To do this, these
plugins need to expand the special items in the above options
and then pass the resulting strings as arguments to job_start().

- Yegappan

[Yegappan Lakshmanan]

Hi Bram,

On Sun, Jun 9, 2019 at 11:19 AM Bram Moolenaar <Br...@moolenaar.net> wrote:
>
> > > > This patch adds a expandcmd() function that can be used to expand all the
> > > > special characters in a command string.
> > >
> > > Thanks. I updated the docs to mention that the expansion happens like
> > > for the ":edit" command. Other commands, such as ":next" work slightly
> > > differently.
> > >

> > > I also reported the error, when there is one. There is still the case
> > > it returns with an error, but without an error message. Perhaps we
> > > should also fail then with some generic message?
> > >
> >
> > Are you referring to the case where expand_filename() returns an
> > empty error message string?
>
> Yes.
>

I went through all the code paths in expand_filename() and the called
functions where an empty error message string is returned. In all these
cases, an error message is already printed. I couldn't find a case
where an empty error message string is returned without printing an
error message. Are you aware of any of theses cases that should be
handled in f_expandcmd()?

Thanks,
Yegappan

[vim-dev ML]

Hi Bram,

On Sun, Jun 9, 2019 at 11:19 AM Bram Moolenaar <Br...@moolenaar.net> wrote:
>

> > > > This patch adds a expandcmd() function that can be used to expand all the
> > > > special characters in a command string.
> > >

> > > Thanks. I updated the docs to mention that the expansion happens like
> > > for the ":edit" command. Other commands, such as ":next" work slightly
> > > differently.
> > >
> > > I also reported the error, when there is one. There is still the case
> > > it returns with an error, but without an error message. Perhaps we
> > > should also fail then with some generic message?
> > >
> >
> > Are you referring to the case where expand_filename() returns an
> > empty error message string?
>
> Yes.
>

I went through all the code paths in expand_filename() and the called
functions where an empty error message string is returned. In all these
cases, an error message is already printed. I couldn't find a case
where an empty error message string is returned without printing an
error message. Are you aware of any of theses cases that should be
handled in f_expandcmd()?

Thanks,
Yegappan

[Bram Moolenaar]

Thanks for checking. I was just wondering if this can be improved.
If it looks like it already works properly, there is no need to spend
more time on this.

--
hundred-and-one symptoms of being an internet addict:

191. You rate eating establishments not by the quality of the food,
but by the availability of electrical outlets for your PowerBook.

[vim-dev ML]

Yegappan wrote:

> On Sun, Jun 9, 2019 at 11:19 AM Bram Moolenaar <Br...@moolenaar.net> wrote:
> >

> > > > > This patch adds a expandcmd() function that can be used to expand all the
> > > > > special characters in a command string.
> > > >

[lacygoill]

@yegappan Thank you for implementing expandcmd(). I would like to use it but I have 2 questions:

Would it make sense for the function to also expand // into /last search pattern/?
https://github.com/vim/vim/blob/7a3330fc578033f06a94c23de61a23edcc59f95e/runtime/doc/quickfix.txt#L1026-L1027

As an example, it could perform this expansion:

/my pattern
:echo expandcmd('vim //gj %')
vim /my pattern/gj current_file

---

Also, expandcmd() does not escape special characters when they're generated by an expansion (e.g. a filename containing the literal character %). For %, #, !, I guess it's not an issue, because we can use fnameescape(); but for spaces, it's different:

$ vim /tmp/foo\ bar/baz
:echo expandcmd('vim /pat/ %')

vim /pat/ /tmp/foo bar/baz

In the expansion, how can we know whether the original command was grepping into the file /tmp/foo bar/baz, or in the files /tmp/foo and $PWD/bar/baz?

Shouldn't expandcmd() at least escape a space when it's used in a filename?

vim /pat/ /tmp/foo\ bar/baz
                  ^

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

[Yegappan Lakshmanan]

Hi,

On Fri, Aug 28, 2020 at 6:49 AM lacygoill <vim-dev...@256bit.org> wrote:

@yegappan Thank you for implementing expandcmd(). I would like to use it but I have 2 questions:

Would it make sense for the function to also expand // into /last search pattern/?
https://github.com/vim/vim/blob/7a3330fc578033f06a94c23de61a23edcc59f95e/runtime/doc/quickfix.txt#L1026-L1027

As an example, it could perform this expansion:

/my pattern
:echo expandcmd('vim //gj %')
vim /my pattern/gj current_file

The expandcmd() function expands the special characters listed under ":h cmdline-special".

As the empty search pattern (//) is not a special character, it won't be expanded.

Just like how commands like 'vimgrep' handle the empty search pattern and replace it

with the last search pattern, the command itself should handle the argument.

---

Also, expandcmd() does not escape special characters when they're generated by an expansion (e.g. a filename containing the literal character %). For %, #, !, I guess it's not an issue, because we can use fnameescape(); but for spaces, it's different:

$ vim /tmp/foo\ bar/baz
:echo expandcmd('vim /pat/ %')

vim /pat/ /tmp/foo bar/baz

In the expansion, how can we know whether the original command was grepping into the file /tmp/foo bar/baz, or in the files /tmp/foo and $PWD/bar/baz?

Shouldn't expandcmd() at least escape a space when it's used in a filename?

vim /pat/ /tmp/foo\ bar/baz
                  ^

I think you can use the ":S" filename modifier to escape the special characters in

the file name.

- Yegappan

[vim-dev ML]

Hi,

On Fri, Aug 28, 2020 at 6:49 AM lacygoill <vim-dev...@256bit.org> wrote:

> @yegappan <https://github.com/yegappan> Thank you for implementing

> expandcmd(). I would like to use it but I have 2 questions:
>
> Would it make sense for the function to also expand // into /last search
> pattern/?
>
> https://github.com/vim/vim/blob/7a3330fc578033f06a94c23de61a23edcc59f95e/runtime/doc/quickfix.txt#L1026-L1027
>
> As an example, it could perform this expansion:
>
> /my pattern
> :echo expandcmd('vim //gj %')
> vim /my pattern/gj current_file
>
>
The expandcmd() function expands the special characters listed under ":h
cmdline-special".
As the empty search pattern (//) is not a special character, it won't be
expanded.
Just like how commands like 'vimgrep' handle the empty search pattern and
replace it
with the last search pattern, the command itself should handle the argument.

------------------------------

>
> Also, expandcmd() does not escape special characters when they're
> generated by an expansion (e.g. a filename containing the literal character
> %). For %, #, !, I guess it's not an issue, because we can use
> fnameescape(); but for spaces, it's different:
>
> $ vim /tmp/foo\ bar/baz
> :echo expandcmd('vim /pat/ %')
>
> vim /pat/ /tmp/foo bar/baz
>
> In the expansion, how can we know whether the original command was
> grepping into the file /tmp/foo bar/baz, or in the files /tmp/foo and
> $PWD/bar/baz?
>
> Shouldn't expandcmd() at least escape a space when it's used in a
> filename?
>
> vim /pat/ /tmp/foo\ bar/baz
> ^
>
>
>
I think you can use the ":S" filename modifier to escape the special
characters in
the file name.

- Yegappan

—
You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or unsubscribe.

[lacygoill]

I think you can use the ":S" filename modifier to escape the special characters in the file name.

Ok, but I don't think it works when expanding ## into the filenames of the arglist.

$ cd /tmp
$ vim a\ b c d\ e
:echo expandcmd('vim /pat/ ##:S')

vim /pat/ a b c d e:S

I guess we need to handle ## specially:

:echo expandcmd('vim /pat/ ' .. argv()->map({_, v -> shellescape(v, 1)})->join())

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.