vim: how to search a pattern through all files in a compressed file (e.g. a tar.gz ball)?

[ping]

assume I have a tar ball containing muliple files and I don't want to
uncompress everything in a folder to start my search (since vim can open
compressed/tar.ed files on the fly), is there way to search a keyword
through all the files in the tarball?

this involves 3 possible solution as I can think of per my limited
knowledge:

1) search everything "on the fly" , this is what I'm asking for, might
not be possible -- those files have not even been opened as a buffer...

2) open from vim everything single files in the tar ball that I want to
search in, then use bufdo? still I don't know of a convenient way of
doing this...

3) of course, if I uncompress everything in a folder, that will be
easier -- external recursive grep can do that I think,

[sc]

On Fri, Sep 14, 2012 at 12:14:41PM -0400, ping wrote:
> assume I have a tar ball containing muliple files and I don't want
> to uncompress everything in a folder to start my search (since vim
> can open compressed/tar.ed files on the fly), is there way to search
> a keyword through all the files in the tarball?

do you have zgrep available? if so you can

zgrep 'pattern' tar-gz-file | vim -

sc

[Ben Fritz]

I didn't know about zgrep! But how about setting 'grepprg' to use it rather than grep? Then you can navigate results easier.

[Tim Chase]

On 09/14/12 12:10, Ben Fritz wrote:
> On Friday, September 14, 2012 12:06:41 PM UTC-5, sc wrote:
>> On Fri, Sep 14, 2012 at 12:14:41PM -0400, ping wrote:
>>> assume I have a tar ball containing muliple files and I don't want
>>> to uncompress everything in a folder to start my search (since vim
>>> can open compressed/tar.ed files on the fly), is there way to search
>>> a keyword through all the files in the tarball?
>>
>> do you have zgrep available? if so you can
>> zgrep 'pattern' tar-gz-file | vim -
>

> I didn't know about zgrep! But how about setting 'grepprg' to use
> it rather than grep? Then you can navigate results easier.

Does it work for you with gzipped tar files? I know you can use
zgrep for plain (non-tar'ed) gzipped files, but when I try it on a
gzipped tar file, it only tells me that a binary file matches:

~$ cd tmp
~/tmp$ mkdir d
~/tmp$ cd d
~/tmp/d$ echo alpha > a.txt
~/tmp/d$ echo beta > b.txt
~/tmp/d$ echo delta > d.txt
~/tmp/d$ cd ..
~/tmp$ tar cvfz d.tgz d/
d/
d/a.txt
d/d.txt
d/b.txt
~/tmp$ zgrep alpha d.tgz
Binary file (standard input) matches

-tim

[sc]

to be honest I don't know -- I learned about zgrep quite recently and
have never had occasion to use it -- I only briefly scanned the man
page before posting -- forgive me, I assumed it would handle tarfiles
too

maybe you'd need something like

tar -xz tar-gz-file | grep 'pattern'

sc

[sc]

and let me add an O:

tar -xzO tar-gz-file | grep 'pattern'

that should do it

sc

[sc]

no, but

tar -zxOf targzfile

will send it to stdout, but grep just reports the binary file matches

so to use vim and search you could

tar -zxOf targzfile | vim -

and then search the buffer for your pattern

sc

[Tony Mechelynck]

You would still have all files concatenated, which might be a little
unwieldy if there are many of them. Maybe untar them to disk (into some
empty ad-hoc directory if the archive doesn't create a top-level
directory), then use the :vimgrep command (q.v.)?

mkdir foobar
tar -zxvC foobar -f rah-rah-blah-604.3.2.en-US.linux-x86_64.tar.gz
gvim
:0verbose vimgrep /pattern/ foobar/**
:cnext
...
(The same works for a .tar.bz2, but with -j instead of -z)

The following mappings may come handy with any quickfix commands:

map <F2> :cnext<CR>
map <S-F2> :cprev<CR>
map <F3> :cnfile<CR>
map <S-F3> :cpfile<CR>
map <F4> :clast<CR>
map <S-F4> :cfirst<CR>


Best regards,
Tony.
--
Have you noticed the way people's intelligence capabilities decline
sharply the minute they start waving guns around?
-- Dr. Who

[ping]

thanks for all the test and response...

in summary here are all suggested/discussed solutions:

1) use zAPPs:
I guess zless, zgrep , zcat, ...these zAPP can't handle tarball, so if I
have a couple of gz files:
a.gz
b.gz
c.gz

they can all do the work in the fly, or redirect to stdout and a "| vim
- "can hand it over.

2) tar with -O (stdout) option, this works fine

tar -zxOf targzfile | vim -

but the issue is (tony pointed out): it looks all file contents are
concatenated, it's not easy to find out from what file the match happens

3) set grepgrp to zgrep
I'm not sure how to make use of this?
currently I can open .gz, or tar.gz with vim directly already
vim abc.tar.gz

4) untar/unzip into flat files into a folder, then use internal/external
tools to recursively search within the folder.

a) external grep:
grep! -R "pattern" . this prints the matches and also gives a quickfix
windows.

b) internal grep:
vimgrep /pattern/ foobar/**


I'll rate solution 3) the best, so it falls into category 3 of following:

this involves 3 possible solution as I can think of per my limited
knowledge:

1) search everything "on the fly", without download and extract tarball
locally

vim scp://user@server/
or
vim
:Nread scp://user@server/
browse to locate the tarball
open it
search the whole tarball

2) open from vim everything single files in the tar ball that I want to
search in, then use bufdo? still I don't know of a convenient way of
doing this...

vim scp://user@server/
or
vim
:Nread scp://user@server/
browse to locate the tarball
open it
open each files in the tarball (turn them into local buffers)
bufdo ...

3) of course, if I uncompress everything in a folder, that will be
easier -- external recursive grep can do that I think,

this is our current solution after download/extract files to a folder:
a) external grep:
grep! -R "pattern" . this prints the matches and also gives a quickfix
windows.

b) internal grep:
vimgrep /pattern/ foobar/**


anyway:
is it possible to get a solution in 1) or 2), that we can get same work
done for remote tarball file, from within vim? ideally I assume this
should be done in netrw if it is possible...
let me copy dr. chip on this...

[ping]

On 09/14/2012 03:19 PM, Tony Mechelynck wrote:

> You would still have all files concatenated, which might be a little
> unwieldy if there are many of them.

b.t.w
I think in non-vim tools zless (or gzip -cd *.gz | less , equivalently)
is super on this . it has the file-spanning next-search (esc-n) that
give you the filename hint about current file that the match is located.

looks for the *.gz files, the best solution is (if netrw don't have sth
fancy), maybe the zless :

login to remote server (telnet/ssh)
zless *.gz
/pattern to define search
n to go next match
esc n to go to next match, spanning all files that contains the match
c-g to see what file we are in
v export (only) current file into vim

for tar.gz file, looks still no better way to figure out which file we
are in, on the fly...