Display shell output in vim split window

[Cory Echols]

Hi all,

I can't get ex-command expansion of the '%' token to work with this
vim tip: http://vim.wikia.com/wiki/Display_shell_commands%27_output_on_Vim_window.

I've put the vimscript from the tip into a file named "test.vim" and
run vim with

vim --noplugin -u test.vim -U NONE test.txt

I would now expect ":Shell cat %" to display the text of "test.txt" in
a split window. (The tip wiki page implies that this should work).
However it does not. I get a split window containing only the text
put into the window by the "setline()" call in the linked script.

If I remove "silent" from the invocation of :read, I discover that the
'%' is passed to the shell without being expanded by vim. If I remove
the call to "escape(a:cmdline, '%#')" I get:

Error 499: Empty file name for '%' or '#' only works with ":p:h"

I *think* this means that vim is expanding '%' in the context of the
new scratch buffer instead of the one I was in when issuing the
":Shell" command.

The wiki page gives me the impression that others are having success
with this, so I'm wondering if their options differ from mine. I've
been through the docs for the shell-related options and for :read, but
they don't lead to any solutions. Has anyone else gotten this to
work?

[Charles Campbell]

Perhaps RunView will do what you want...

http://vim.sourceforge.net/scripts/script.php?script_id=2511

If not:

* exe "Shell ".fnameescape(expand("%"))

The reason why "%" wasn't expanded: see the line
execute 'silent $read !'.escape(a:cmdline,'%#')

That escapes all instances of % and/or # .

The help for the use of % (and #) in command lines is at :help
cmdline-special .

Regards,
Chip Campbell

[John Beckett]

Cory Echols wrote:
> I can't get ex-command expansion of the '%' token to work
> with this vim tip:
>
http://vim.wikia.com/wiki/Display_shell_commands%27_output_on_Vim_window
.

Teemu Likonen:
Are you reading this? Please help fix your tip!

Cory:
Thanks for reporting the problem. I haven't tried the script, but a
sanity check immediately fails. Entering the following gives the E499
error you mentioned:

:new
:r !cat %

As you said, that's because the command is executed in a scratch window,
and there is no file name. However, using # will work (assuming the
alternate buffer is the one you want). That is, use # instead of %.

More problems: As Charles said, the escape() is going to cause grief. I
just looked through all the posts on the git mailing list (see the
discussion on the proposed new tips page, via link at top of tip). They
don't seem to think % is a problem, but have various other suggestions.

I don't have time to fix all the new tips that people drop on the Vim
Tips wiki. If we can't get this working before March, I'm going to
suggest it gets removed. If anyone discovers something, please let us
know.

John

[Teemu Likonen]

On 2009-02-03 09:28 (+1100), John Beckett wrote:

> Teemu Likonen:
> Are you reading this? Please help fix your tip!

I'm here. :-)

> More problems: As Charles said, the escape() is going to cause grief.
> I just looked through all the posts on the git mailing list (see the
> discussion on the proposed new tips page, via link at top of tip).
> They don't seem to think % is a problem, but have various other
> suggestions.
>
> I don't have time to fix all the new tips that people drop on the Vim
> Tips wiki. If we can't get this working before March, I'm going to
> suggest it gets removed. If anyone discovers something, please let us
> know.

This can't really be fixed. % and # are supposed to be expanded when
user executes the command line. These characters are being escape()d in
the script so that they don't get expanded twice. Otherwise even already
backslash-escaped % (for example, ":Shell echo \%") would be expanded
(and when % expands to filename with % character). That's because the
"execute" command will again run "!" command and the command line is
expanded again.

The actual problem is complicated. Try this example:

command! -complete=file -nargs=* Test ThisCommandDoesntExist

Now command ":Test %" gives that E499 error when executed in a buffer
with no filename. It never even tries to execute the non-existing
command. If we remove the -complete=file option then the command will be
executed and % and # are not expanded when user executes the command
line.

In theory those Vim meta characters could be expanded in the
s:RunShellCommand() function when it runs "!" command with "execute".
The problem is now that % and # are expanded in the context of the
scratch buffer and hence % doesn't actually point to anything useful. So
we want to keep the expand(a:cmdline,'%#') in the script untouched.

I'll conclude that we can have a sort of fix to this but we lose
filename completion and we lose % and # expansion. They are too useful
to me so I'm not going to change the version of the script on my
computer but for the Vim tip page the fix would be to remove the
-complete=file option:

command! -nargs=+ Shell call s:RunShellCommand(<q-args>)

And we need to remove the examples with % expansion.

[Teemu Likonen]

On 2009-02-03 07:48 (+0200), Teemu Likonen wrote:

> [...] for the Vim tip page the fix would be to remove the

> -complete=file option:
>
> command! -nargs=+ Shell call s:RunShellCommand(<q-args>)
>
> And we need to remove the examples with % expansion.

...or we just tell people that % expansion doesn't work with empty
buffers. Vim prints the error E499 when parsing the command line, not
when executing the commands in s:RunShellCommand().

[John Beckett]

Teemu Likonen wrote:
> This can't really be fixed. % and # are supposed to be
> expanded when user executes the command line. These
> characters are being escape()d in the script so that they
> don't get expanded twice.

Thanks for joining in. I'm running on low battery and can't quite follow
the details at the moment. However, supposing I understood this, I still
don't see why you would escape % and #. Are you doing it in case you
want to use those characters on the shell command line (that is, nothing
to do with the Vim meanings for % and #)? If so, wouldn't there be a
bunch of other characters that needed escaping??

> I'll conclude that we can have a sort of fix to this but we
> lose filename completion and we lose % and # expansion. They
> are too useful to me so I'm not going to change the version
> of the script on my computer but for the Vim tip page the fix
> would be to remove the -complete=file option:
>
> command! -nargs=+ Shell call s:RunShellCommand(<q-args>)
>
> And we need to remove the examples with % expansion.

I agree that filename completion is too useful to discard. Again, I'm
confused by how the above seems to imply that % and # do something
useful for you, yet % failed for Cory.

Being really brutal about this, while I may one day want this tip for my
personal work, at the moment all I want to do is have someone clean up
the proposed new tip so that it makes sense and works as advertised. I
know that in the past we've had several inclusionists who want
everything retained on the wiki, but those of us who actually do the
retaining are sufficiently engaged with the old tips, and we want new
tips to be reasonably clean (or at least to have a "todo" to make any
shortcomings apparent).

John

[Teemu Likonen]

On 2009-02-03 18:52 (+1100), John Beckett wrote:

> However, supposing I understood this, I still don't see why you would
> escape % and #.

Put this code to a file and source it:

command! -complete=file -nargs=* Test call s:TestFunc(<q-args>)

function! s:TestFunc(args)
echo a:args
execute '!ls -l ' . a:args
endfunction

Now select a buffer which has a file that exists in the filesystem. Run
these commands:

:Test %
:Test \%

Besides echoing the a:args both will print the "ls -l" for that
filename, like this:

-rw-r--r-- 1 dtw dtw 132 3.2. 10:03 test.vim

In the ":Test %" case the % is expanded when Vim parses the command
line. In the ":Test \%" case Vim command line parser removes the
backslash and passes "%" to the function in which it is expanded by the
"!ls -l" command.

So we really just want these meta characters expanded only on the
command line. That's why we escape them in the function. Think about a
situation where the filename itself contains a "%" character. Then the
result of ":Test %" is like the following. We use filename "test-%.vim"
here.

First ":Test %" echoes a:args, that is, the % expanded to the filename:

test-%.vim

Then we see an error from "ls" command:

:!ls -l test-test-%.vim.vim
ls: cannot access test-test-%.vim.vim: No such file or directory

A user of commands like :Test or :Shell deals directly with the command
line so expanding Vim meta characters only in that context is the right
thing to do. Vim does that automatically.

Have I convinced everybody that we don't want to expand "%" and "#"
twice?

> [...] wouldn't there be a bunch of other characters that needed
> escaping??

Yes. In shell, parentheses, for example, would need escaping too.
Suppose we have a filename which has parentheses, like "test-().vim".
Let's try ":Test %" command again:

test-().vim
:!ls -l test-().vim
/bin/bash: -c: line 0: syntax error near unexpected token `('
/bin/bash: -c: line 0: `ls -l test-().vim'

The first line is the output of "echo a:args". Parentheses are shell
meta characters and the would need escaping in shell's context.

On the other hand "%" and "#" need escaping as Vim's context. But we
can't use "escape(a:cmdline, '%#()')" (that is, escape the parentheses
too) because then we couldn't pass parentheses as themselves anymore
from the Vim command line. The problem comes from the fact that there
are three levels where expansion happens:

1. when user executes the :Test or :Shell command (Vim expansion)
2. when the script runs "!" command (Vim expansion)
3. when the shell runs the command line (shell expansion).

Vim's expanding/escaping business is such a mess and it just can't work
in all situations. It has bitten me many times even though I think I
understand it quite well.

So after trying different things I think the best compromize is the one
already in the Vim tip page.

>> I'll conclude that we can have a sort of fix to this but we
>> lose filename completion and we lose % and # expansion. They
>> are too useful to me so I'm not going to change the version
>> of the script on my computer but for the Vim tip page the fix
>> would be to remove the -complete=file option:
>>
>> command! -nargs=+ Shell call s:RunShellCommand(<q-args>)
>>
>> And we need to remove the examples with % expansion.
>
> I agree that filename completion is too useful to discard. Again, I'm
> confused by how the above seems to imply that % and # do something
> useful for you, yet % failed for Cory.

I'm confused too. If the currently active buffer has a filename, then
with :Shell command "%" and "#" are expanded correctly. They work well
here. If the buffer don't have a filename then Vim (quite rightly) gives
E499 error without even executing the custom :Shell command.

> [...] at the moment all I want to do is have someone clean up the

> proposed new tip so that it makes sense and works as advertised.

Here it does work as advertised. It's the Vim command-line parser that
gives the E499 error. It happens before my s:RunShellCommand function is
even called.

[Cory Echols]

After some tinkering, I came up with this version, which attempts to
expand tokens mentioned in ':he cmdline-special' before moving the
cursor to the scratch buffer:

command! -complete=shellcmd -nargs=+ Shell call s:RunShellCommand(<q-args>)
function! s:RunShellCommand(cmdline)
echo a:cmdline
let expanded_cmdline = a:cmdline
for part in split(a:cmdline, ' ')
if part[0] =~ '\v[%#<]'
let expanded_part = fnameescape(expand(part))
let expanded_cmdline = substitute(expanded_cmdline, part,
expanded_part, '')
endif
endfor
botright new
setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
call setline(1, 'You entered: ' . a:cmdline)
call setline(2, 'Expanded Form: ' .expanded_cmdline)
call setline(3,substitute(getline(2),'.','=','g'))
execute '$read !'. expanded_cmdline
setlocal nomodifiable
1
endfunction

It likely has problems of its own. It assumes everything that will be
expand()'ed will result in a file name, for one thing. However, I can
get trivial buffer name expansions to work, so I'm happy.