Using :g to filter lines through an external command

[A. Wik]

Hi all,

Assume I have a file (or just a buffer) with the contents:
echo Hello from /bin/sh
test1
test2
test3
---end-file---

:1!sh does what I would expect: it passes "echo Hello from /bin/sh" as
input to the shell, which executes the line as a command, and the line
is replaced with "echo"'s output "Hello from /bin/sh".

:g/test/!nl does not do what I expected (and wanted). Instead of
piping the matched lines through the "nl" command, it appears to start
an interactive shell.

:g/test/.!nl works better, but executes "nl" once for *each* of the
matched lines, which are replaced with the following:
1 test1
1 test2
1 test3
---end-quote---

Is there a solution to this?

-Albert.

[Tim Chase]

On 2020-07-12 09:59, A. Wik wrote:
> Hi all,
>
> Assume I have a file (or just a buffer) with the contents:
> echo Hello from /bin/sh
> test1
> test2
> test3
> ---end-file---
>
> :1!sh does what I would expect: it passes "echo Hello from /bin/sh"
> as input to the shell, which executes the line as a command, and
> the line is replaced with "echo"'s output "Hello from /bin/sh".
>
> :g/test/!nl does not do what I expected (and wanted). Instead of
> piping the matched lines through the "nl" command, it appears to
> start an interactive shell.

That is nl(1) waiting for input. Sending an EOF (control+D) will
terminate it (note that because it gets invoked 3 times, one for each
line, you'll need to ^D 3x as well).

> :g/test/.!nl works better, but executes "nl" once for *each* of the
> matched lines, which are replaced with the following:
> 1 test1
> 1 test2
> 1 test3
> ---end-quote---
>
> Is there a solution to this?

Well, for this particular buffer, you're processing the whole file so
you could just use

:%!nl

which will do what you want.

However if you have disjoint ranges, you'll need to go a bit further.
If you want contiguous ranges like

test1
test2
not this one
test3
test4
not this one either

to be numbered

1 test1
2 test2
not this one
1 test3
2 test4
not this one either

you might do something like

:g/^test/.,/^\(test\)\@!/-!nl

which looks for lines starting with "test" then, starting a range
there ('.') through (',') the line before ('-') the next line that
doesn't start with "test" ("/^\(test\)\@!/"), runs each of those
ranges through nl(1)

If you want nl(1) to jump gaps across disjoint ranges, you'll have to
change strategy. I'd use vim's in-built expression-evaluation:

:let i=0 | g/^test/let i+=1 | s/^/\=i.' '

(adjust formatting to taste).

-tim

[A. Wik]

On Sun, 12 Jul 2020 at 19:12, Tim Chase <v...@tim.thechases.com> wrote:
>
> On 2020-07-12 09:59, A. Wik wrote:
> > Hi all,
> >
> > Assume I have a file (or just a buffer) with the contents:
> > echo Hello from /bin/sh
> > test1
> > test2
> > test3
> > ---end-file---
>

> Well, for this particular buffer, you're processing the whole file so
> you could just use
>
> :%!nl
>
> which will do what you want.

2,$!nl actually -- there's an "echo" line first.

> you might do something like
>
> :g/^test/.,/^\(test\)\@!/-!nl
>
> which looks for lines starting with "test" then, starting a range
> there ('.') through (',') the line before ('-') the next line that
> doesn't start with "test" ("/^\(test\)\@!/"), runs each of those
> ranges through nl(1)

That is an improvement.

> If you want nl(1) to jump gaps across disjoint ranges, you'll have to
> change strategy. I'd use vim's in-built expression-evaluation:
>
> :let i=0 | g/^test/let i+=1 | s/^/\=i.' '

Good.

You can use an external program to do the work; eg:
:%!awk 'BEGIN { i=1 } /^test/ { printf "\%d \%s\n", i++, $0; next; }
{ print $0 }'

What is the best way for collecting the output of :g? This works:
:g/^test/.w! >> test.tmp
But it does require a temporary file. Why is the dot required?
Leaving it out produces a test.tmp containing several repetitions of
the whole source file.

:let @a = "" | g/^test/y A
also works, but it puts an empty line first and last in the A
register; of course, deleting them is easy enough.

:v/^test/d
works too; you can use undo to get the other lines back.

:let @a="" | redir @A | g/^test/
:redir END
produces an A register like the "y A" above, with a blank line at the
start and end.

-aw