[vim/vim] allow Vim script expressions to be evaluated inside heredocs (Issue #10129)

lacygoill

unread,

Apr 9, 2022, 4:20:19 AM4/9/22

to vim/vim, Subscribed

Is your feature request about something that is currently impossible or hard to do? Please describe the problem.

It is not easy to include the value of a Vim script expression inside a heredoc:

vim9script

var lines =<< trim END

    echo $HOME

END

echo lines

['echo $HOME']

If we want $HOME to be evaluated, we need an extra call to a combination of map() and substitute():

vim9script

var lines =<< trim END

    echo $HOME

END

lines->map((_, line) => line->substitute('$HOME', $HOME, ''))

echo lines

[echo /home/user]

Which makes the code less readable, and is probably not always reliable in the general case (i.e. substitute() might replace tokens which we might want to be left alone).

Describe the solution you'd like

Extend the backtick expansion syntax which supports Vim script expressions (the one documented at :help E1083) to heredocs:

:e `=tempname()`

   ^^          ^

With this feature, the previous snippet could be simplified into this:

vim9script

var lines =<< trim eval END

    echo `=$HOME`

END

echo lines

Notice the presence of the new eval argument on the first line of the assigment:

var lines =<< trim eval END

                   ^--^

It would be necessary to avoid breaking existing scripts. With eval, Vim would look for Vim script expressions to evaluate.

Describe alternatives you've considered

An alternative is to continue using an extra call to a combination of map() and substitute(), which – again – is less readable and less reliable than the proposed syntax.

Additional context

In bash, there are 2 kinds of heredocs. The ones which do not try to expand any parameter:

cat <<'EOF'

    echo $HOME

EOF

echo $HOME

And the ones which do:

cat <<EOF

    echo $HOME

EOF

echo /home/user

The difference between the 2 depends on whether the end word is quoted. Currently, Vim script only supports the first kind. I think it would be useful to support the second one too.

—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Apr 9, 2022, 6:57:30 AM4/9/22

to vim/vim, Subscribed

I think we can leave out the equal sign, just use backticks around the expression.
We can also say that two consecutive backticks stand for one, so that you can still use them without being evaluated.

It looks useful, but before implementing this we should have a discussion about alternatives.
Will this method always work? Is there a better way?

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

lacygoill

unread,

Apr 9, 2022, 7:22:05 AM4/9/22

to vim/vim, Subscribed

I think we can leave out the equal sign, just use backticks around the expression.

We could but:

we would not be able to use simple backticks to later support shell commands (although =system() would be a workaround)
it would be inconsistent with how backticks work on the command-line (e.g. after :edit)

Although, I admit it would be more convenient (because in practice, we would probably need to evaluate a Vim script expression more often than a shell command).

We can also say that two consecutive backticks stand for one, so that you can still use them without being evaluated.

That looks a bit weird. Usually, when a character has a special meaning and we want it to be literal, we escape it (usually with a backslash).

Will this method always work? Is there a better way?

FWIW, UltiSnips (which is one of the most popular snippets plugin) uses a similar syntax to let the user evaluate code inside snipppets.

Inside a snippet a snippet, this is shell code:

`shell command`

This is Vim script code:

`!v Vim script expression`

This is Python code:

`!p python statement`

And this is a literal backtick:

\`

I think other snippets plugins use a similar kind of syntax.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

LemonBoy

unread,

Apr 9, 2022, 8:58:18 AM4/9/22

to vim/vim, Subscribed

The use of backticks a-la JS templated strings was already discussed (and implemented in #4634), expanding the logic to work in heredocs may help solve this problem.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Apr 9, 2022, 11:09:17 AM4/9/22

to vim/vim, Subscribed

> > I think we can leave out the equal sign, just use backticks around the expression.
>
> We could but:
>

> - we would not be able to use simple backticks to later support

> shell commands (although `=system()` would be a workaround)

Using shell commands would be (or should be) rare. and you can use
`system(cmd)`.

> - it would be inconsistent with how backticks work on the
> command-line (e.g. after `:edit`)

True. But then backtick without equal sign would mean using a shell
command. Which is really slow and has escaping problems.

> Although, I admit it would be more convenient (because in practice, we
> would probably need to evaluate a Vim script expression more often
> than a shell command).

Right.

> > We can also say that two consecutive backticks stand for one, so that you can still use them without being evaluated.
>
> That looks a bit weird. Usually, when a character has a special
> meaning and we want it to be literal, we escape it (usually with a
> backslash).

The problem with using a backslash is that this makes backslash a
special character, then you would need to double a backslash to get one.
Otherwise, how would you get a literal \` ? And backslashes are used
often, while if we only make backtick special it's a lot simpler.

Compare this with single quoted strings, where the only the single quote
is special, needs to be doubled to get one. Heredocs are similar,
in that nearly everything is taken literally.

--
The problem with political jokes is that they get elected.

/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Yegappan Lakshmanan

unread,

Apr 9, 2022, 8:39:54 PM4/9/22

to vim_dev, reply+ACY5DGBSK3M7F4XGKL...@reply.github.com, vim/vim, Subscribed

Hi,

I have created PR https://github.com/vim/vim/pull/10138 with the support for this.

I have tested this only with the legacy script. Can you try this out and let me know if

you see any issues?

Thanks,

Yegappan

lacygoill

unread,

Apr 17, 2022, 8:02:21 AM4/17/22

to vim/vim, Subscribed

Fixed by patch 8.2.4770

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

lacygoill

unread,

Apr 17, 2022, 8:02:22 AM4/17/22

to vim/vim, Subscribed

Closed #10129.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

vim-dev ML

unread,

Oct 11, 2022, 4:26:05 AM10/11/22

to vim/vim, vim-dev ML, Your activity

Hi,

On Sat, Apr 9, 2022 at 1:20 AM lacygoill ***@***.***> wrote:

> *Is your feature request about something that is currently impossible or
> hard to do? Please describe the problem.*

>
> It is not easy to include the value of a Vim script expression inside a
> heredoc:
>
> vim9script
> var lines =<< trim END
>
> echo $HOME
>
> END
> echo lines
>
> ['echo $HOME']
>
>
> If we want $HOME to be evaluated, we need an extra call to a combination
> of map() and substitute():
>
> vim9script
> var lines =<< trim END
>
> echo $HOME
>
> END
> lines->map((_, line) => line->substitute('$HOME', $HOME, ''))
> echo lines
>
> [echo /home/user]
>
>
> Which makes the code less readable, and is probably not always reliable in
> the general case (i.e. substitute() might replace tokens which we might
> want to be left alone).
>

> *Describe the solution you'd like*

>
> Extend the backtick expansion syntax which supports Vim script expressions
> (the one documented at :help E1083

> <https://vimhelp.org/editing.txt.html#E1083>) to heredocs:

>
> :e `=tempname()`
>
> ^^ ^
>
>
> With this feature, the previous snippet could be simplified into this:
>
> vim9script
> var lines =<< trim eval END
>
> echo `=$HOME`
>
> END
> echo lines
>
> Notice the presence of the new eval argument on the first line of the
> assigment:
>
> var lines =<< trim eval END
>
> ^--^
>
>
> It would be necessary to avoid breaking existing scripts. With eval, Vim
> would look for Vim script expressions to evaluate.
>

> *Describe alternatives you've considered*

>
> An alternative is to continue using an extra call to a combination of
> map() and substitute(), which – again – is less readable and less
> reliable than the proposed syntax.
>

> *Additional context*

>
> In bash, there are 2 kinds of heredocs. The ones which do not try to
> expand any parameter:
>
> cat <<'EOF'
> echo $HOME
> EOF
>
> echo $HOME
>
>
> And the ones which do:
>
> cat <<EOF
> echo $HOME
> EOF
>
> echo /home/user
>
>
> The difference between the 2 depends on whether the end word is quoted.
> Currently, Vim script only supports the first kind. I think it would be
> useful to support the second one too.
>
>
>

I have created PR https://github.com/vim/vim/pull/10138 with the support
for this.
I have tested this only with the legacy script. Can you try this out and
let me know if
you see any issues?

Thanks,
Yegappan

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

Reply all

Reply to author

Forward