Gnu Emacs Bug: rgrep, “find: invalid predicate `-nam'”

[Xah Lee]

Gnu Emacs Bug: rgrep, “find: invalid predicate `-nam'”

this seems to be a bug. Steps to reproduce:

Start GNU Emacs with -Q.
call “rgrep”. Search for anything.
You get the following result:

-*- mode: grep; default-directory: "c:/Users/h3/web/xahlee_org/emacs/"
-*-
Grep started at Thu Nov 17 00:49:37

find . "(" -path "*/SCCS" -o -path "*/RCS" -o -path "*/CVS" -o -path
"*/MCVS" -o -path "*/.svn" -o -path "*/.git" -o -path "*/.hg" -o -path
"*/.bzr" -o -path "*/_MTN" -o -path "*/_darcs" -o -path "*/{arch}" ")"
-prune -o "(" -name ".#*" -o -name "*.o" -o -name "*~" -o -name
"*.bin" -o -name "*.bak" -o -name "*.obj" -o -name "*.map" -o -name
"*.ico" -o -name "*.pif" -o -name "*.lnk" -o -name "*.a" -o -name
"*.ln" -o -name "*.blg" -o -name "*.bbl" -o -name "*.dll" -o -name
"*.drv" -o -name "*.vxd" -o -name "*.386" -o -name "*.elc" -o -name
"*.lof" -o -name "*.glo" -o -name "*.idx" -o -name "*.lot" -o -name
"*.fmt" -o -name "*.tfm" -o -name "*.class" -o -name "*.fas" -o -name
"*.lib" -o -name "*.mem" -o -name "*.x86f" -o -name "*.sparcf" -o -
name "*.fasl" -o -name "*.ufsl" -o -name "*.fsl" -o -name "*.dxl" -o -
name "*.pfsl" -o -name "*.dfsl" -o -name "*.p64fsl" -o -name
"*.d64fsl" -o -name "*.dx64fsl" -o -name "*.lo" -o -name "*.la" -o -
name "*.gmo" -o -name "*.mo" -o -name "*.toc" -o -name "*.aux" -o -
name "*.cp" -o -name "*.fn" -o -name "*.ky" -o -name "*.pg" -o -name
"*.tp" -o -name "*.vr" -o -name "*.cps" -o -name "*.fns" -o -name
"*.kys" -o -name "*.pgs" -o -name "*.tps" -o -name "*.vrs" -o -name
"*.pyc" -o -name "*.pyo" ")" -prune -o -type f "(" -iname "*" -o -
iname ".*" ")" -exec grep -i -n "grep" {} NUL ";"
FIND: Parameter format not correct

Grep exited abnormally with code 2 at Thu Nov 17 00:49:37



This is GNU Emacs version “GNU Emacs 23.3.1 (i386-mingw-nt6.1.7601) of
2011-03-10 on 3249CTO”.

If you are using ErgoEmacs with -Q, you get this extra info “find:
invalid predicate `-nam'”, like this:


-*- mode: grep; default-directory: "c:/Users/h3/web/xahlee_org/emacs/"
-*-
Grep started at Thu Nov 17 00:36:51

find . "(" -path "*/SCCS" -o -path "*/RCS" -o -path "*/CVS" -o -path
"*/MCVS" -o -path "*/.svn" -o -path "*/.git" -o -path "*/.hg" -o -path
"*/.bzr" -o -path "*/_MTN" -o -path "*/_darcs" -o -path "*/{arch}" ")"
-prune -o "(" -name ".#*" -o -name "*.beam" -o -name "*.vee" -o -name
"*.jam" -o -name "*.o" -o -name "*~" -o -name "*.bin" -o -name "*.bak"
-o -name "*.obj" -o -name "*.map" -o -name "*.ico" -o -name "*.pif" -o
-name "*.lnk" -o -name "*.a" -o -name "*.ln" -o -name "*.blg" -o -name
"*.bbl" -o -name "*.dll" -o -name "*.drv" -o -name "*.vxd" -o -name "*.
386" -o -name "*.elc" -o -name "*.lof" -o -name "*.glo" -o -name
"*.idx" -o -name "*.lot" -o -name "*.fmt" -o -name "*.tfm" -o -name
"*.class" -o -name "*.fas" -o -name "*.lib" -o -name "*.mem" -o -name
"*.x86f" -o -name "*.sparcf" -o -name "*.fasl" -o -name "*.ufsl" -o -
name "*.fsl" -o -name "*.dxl" -o -name "*.pfsl" -o -name "*.dfsl" -o -
name "*.p64fsl" -o -name "*.d64fsl" -o -name "*.dx64fsl" -o -name
"*.lo" -o -name "*.la" -o -name "*.gmo" -o -name "*.mo" -o -name
"*.toc" -o -name "*.aux" -o -name "*.cp" -o -name "*.fn" -o -name
"*.ky" -o -name "*.pg" -o -name "*.tp" -o -name "*.vr" -o -name
"*.cps" -o -name "*.fns" -o -name "*.kys" -o -name "*.pgs" -o -name
"*.tps" -o -name "*.vrs" -o -name "*.pyc" -o -name "*.pyo" ")" -prune -
o -type f "(" -iname "*.html" ")" -print0 | xargs -0 -e grep -i -nH -
e "grep"
find: invalid predicate `-nam'

Grep finished with no matches found at Thu Nov 17 00:36:51



Seems quite a odd error. It appears that there's a typo somewhere that
contains -nam instead of -name, but i wasn't able to find it anywhere
so far. Note that this error occurs when starting emacs with -Q, so it
rules out personal init files.

Xah Lee

[blandest]

You are probably calling the wrong `find'. Instead of the GNU find you
are most likely using the one built in Windows' cmd.exe.
If this is indeed the cause, you will need to adjust your PATH
environment or put an appropriate find.exe in the System dir specific
to your Windows version.

[Xah Lee]

On Nov 17, 4:09 am, blandest <valentin.ba...@gmail.com> wrote:
> On Nov 17, 11:19 am, Xah Lee <xah...@gmail.com> wrote:
> > Gnu Emacs Bug: rgrep, “find: invalid predicate `-nam'”
>
> > this seems to be a bug. Steps to reproduce:
>
> >     Start GNU Emacs with -Q.
> >     call “rgrep”. Search for anything.
> >     You get the following result:
>
> > -*- mode: grep; default-directory: "c:/Users/h3/web/xahlee_org/emacs/"
> > -*-
> > Grep started at Thu Nov 17 00:49:37
>
> > find . "(" -path "*/SCCS" -o -path "*/RCS" -o -path "*/CVS" -o -path

> > ...

> > "*.pyc" -o -name "*.pyo" ")" -prune -o  -type f "(" -iname "*" -o -
> > iname ".*" ")" -exec grep -i -n "grep" {} NUL ";"
> > FIND: Parameter format not correct

> > ...
> > find: invalid predicate `-nam'

>

> You are probably calling the wrong `find'. Instead of the GNU find you
> are most likely using the one built in Windows' cmd.exe.
> If this is indeed the cause, you will need to adjust your PATH
> environment or put an appropriate find.exe in the System dir specific
> to your Windows version.

Thanks. Great answer.

That seems to lead somewhere. After i manually set PATH and exec-path
(probably just the latter is necessary), then the problem went away,
but a different problem occurs.

In the output, for each file that doesn't contains the search text, it
shows:

grep: NUL: No such file or directory
grep: NUL: No such file or directory
grep: NUL: No such file or directory
...

it's curious, today i was writing a tutorial on all the ways to search
text in files.
occur (alias list-matching-lines), grep, rgrep, lgrep, find-dired, ...
it's somewhat inconsistent.

list-matching-lines uses elisp, while the others relies on unix util.

The interface also isn't consistent. e.g. grep and grep-find (alias
find-grep) both directly prompt you to enter unix command in one shot.
But find-dired, rgrep, lgrep, do several prompts asking for search
string, file extension, then dir (though, still based on unix command
syntax as they are just passed to unix shell. e.g. when asking file
extension, you use unix wildcard such as *html instead of regex.)

also, occur shows result with search term highlighted. Nice. But grep
command doesn't.

It seems to me, they could all use elisp, with a single code engine,
and with the same interface. The files to be searched can be from
buffer list or dired marked files, or entered from prompt with emacs
regex.
The output can be a output like occur or can be dired list or buffer
list.
e.g. list-matching-lines, and then
list-matching-lines-directory with options to do nested dirs, and
options to use a pattern to filter files to search for, or options to
get marked files in dired, and also options to show result as list in
dired.

has someone written multi-file grep in elisp? am thinking that someone
must have.

Xah

[blandest]

Both 'grep' and 'rgrep' commands show me a nice buffer with
highlighted matching results. I am however using Emacs 24 on Linux.
You may want to try with other color themes, maybe the highlighting
color is not customized accordingly in your setup.

On your 'Unix utils' vs Elisp stuff... There is a good idea to rely on
(excellent) external utilities for both the flexibility and the
performance that they come with. These tools search through files that
may not necessarily contain the expected pattern or that do not have
a file type you would normally edit in Emacs. You gain some speed if
you do not recursively open all files in a directory for which you
need to have the corresponding major and minor modes loaded, hooks
triggered (and so on), after you which you can begin to search for the
desired pattern. (There may be a way to open a file without activating
major modes or triggering hooks, but I haven't looked into it yet).

However, functions like 'list-matching-lines' are better implemented
in Emacs Lisp since their job is much simpler.

You can write your own Elisp wrapper functions that run interactively
with the purpose of reducing the number of parameters you are prompted
for. I did this once but can't remember why :)

[Stephen Powell]

Xah Lee <xah...@gmail.com> writes:

> In the output, for each file that doesn't contains the search text, it
> shows:
>
> grep: NUL: No such file or directory
> grep: NUL: No such file or directory
> grep: NUL: No such file or directory

NUL is the name cmd.exe uses for the null device. If your using a unixy
shell try (setq null-device "/dev/null").

[Stefan Monnier]

> grep: NUL: No such file or directory
> grep: NUL: No such file or directory
> grep: NUL: No such file or directory
> ...

Are you running Cygwin's `find' rather than a native w32 port, by any chance?


Stefan

[Stefan Monnier]

BTW, assuming Cygwin's `find' works well once you (setq null-device
"/dev/null"), it would probably make sense for Emacs to not use NUL when
find is Cygwin's. So please M-x report-emacs-bug with as much detail so
we can try and see how/if to auto-detect this case.


Stefan

[Xah Lee]

yes, Cygwin, sometimes MinGW. Tried a couple of times to use those w32
port of unix tool but never got them to work...

Xah

[Xah Lee]

On Nov 17, 8:48 am, blandest <valentin.ba...@gmail.com> wrote:
> Both 'grep' and 'rgrep' commands show me a nice buffer with
> highlighted matching results. I am however using Emacs 24 on Linux.

humm... is this new in emacs 24 or is it because it's emacs on linux?

> You may want to try with other color themes, maybe the highlighting
> color is not customized accordingly in your setup.

nah, it doesn't have to do with any personal customization.

> On your 'Unix utils' vs Elisp stuff... There is a good idea to rely on
> (excellent) external utilities for both the flexibility and the
> performance that they come with. These tools search through files that
> may not necessarily contain the expected pattern or that do not have
> a  file type you would normally edit in Emacs. You gain some speed if
> you do not recursively open all files in a directory for which you
> need to have the corresponding major and minor modes loaded, hooks
> triggered (and so on), after you which you can begin to search for the
> desired pattern. (There may be a way to open a file without activating
> major modes or triggering hooks, but I haven't looked into it yet).
>
> However, functions like 'list-matching-lines' are better implemented
> in Emacs Lisp since their job is much simpler.
>
> You can write your own Elisp wrapper functions that run interactively
> with the purpose of reducing the number of parameters you are prompted
> for. I did this once but can't remember why :)

Calling external util has lots problems, especially under Windows.
First of all, on Windows, external util may not be installed. I find
this to be a show stopper. Then, there's Cygwin, MinGW and other
variety issues and the complex interface to these programs thru
several layers. On unix/linux including Mac OS X, there's rather
complex issues of identifying the version of grep/find used, which
differ in options and behavior. I recall than “rgrep” didn't work on
Mac OS X too. Then, emacs has to process the output to make syntax
coloring. Also, if your search string contains Unicode, then there's
extremely complex problem about setting various parameters or
environment variables about encoding thru different layers.

On the other hand, these text searching task is what elisp is designed
to do, is rather trivial to code. It would eliminate the whole
external program problems.

By the way, doing it inside elisp is not that slow. Slower than those
written in C, but remeber that emacs has to parse their output and the
communication overhead might just make it up. I've written a grep
command in elisp ( http://xahlee.org/emacs/emacs_grep_find.html ). My
script is more or less equivalent to unix's 「grep -c」. Just tested
now, calling it on a dir with subdir with total of 1592 files. Both
are close to 3 seconds. (e.g. call “shell” in emacs, then give “grep -
c */*html”) Calling “grep -c */*html” by “shell-command” is less than
1 second. Calling “grep -c */*html” by emacs's “grep” is about 3
seconds too.

We can set up a more precise and controlled test, ... but overall i
don't think having grep commands implemented in emacs makes difference
in speed for daily practical use. If user wanted to process tens of
thousand files or large sized files, they can still call unix's shell
commands directly if that is any faster. I process 5 thousand files
about every day, using various elisp script i've written.

Xah

[Xah Lee]

correction. The link should be http://xahlee.org/emacs/elisp_grep_script.html

Xah

[blandest]

On Nov 18, 9:11 pm, Xah Lee <xah...@gmail.com> wrote:
> correction. The link should behttp://xahlee.org/emacs/elisp_grep_script.html
>
>  Xah

`with-temp-file' seems to be the alternative of directly opening the
file and it is indeed faster.

When 'rgrep' is not available you can probably call 'grep -r'. I had
this issue on a RedHat Linux system, but this was a shell thing. Emacs
calls 'find' and then grep on the matched files as far as I know, so
invoking 'M-x rgrep' should not be a problem even if you do not have
'rgrep' installed.

The 'grep script' you have linked to seems enough for what you want to
accomplish, but it does not have the same flexibility as the Emacs'
rgrep command does and it would take considerable additional effort if
you want to make it more generic and usable for most people. Also,
implementing it in Emacs Lisp limits the user to the supported regular
expression style, making it impossible to use the Perl compatible one.

Just curious about the Unicode issues that you had, have you tried to
customize the variable 'grep-command' which defaults to "grep -nH -e "
in my setup, and pass in the "-P" parameter (for Perl regex) and then
search for a string containing an Unicode pattern? I am usually
grepping for English text in source code files so I did not run into
this kind of issues.

[Xah Lee]

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10074

[Xah Lee]

On Nov 18, 12:55 pm, blandest <valentin.ba...@gmail.com> wrote:
> On Nov 18, 9:11 pm, Xah Lee <xah...@gmail.com> wrote:
> > correction. The link should be http://xahlee.org/emacs/elisp_grep_script.html
>

> `with-temp-file' seems to be the alternative of directly opening the
> file and it is indeed faster.

yeah. Am still horning my elisp skills... recently tried to looked at
how dired-do-query-replace-regexp do it...

> When 'rgrep' is not available you can probably call 'grep -r'. I had
> this issue on a RedHat Linux system, but this was a shell thing. Emacs
> calls 'find' and then grep on the matched files as far as I know, so
> invoking 'M-x rgrep' should not be a problem even if you do not have
> 'rgrep' installed.
>
> The 'grep script' you have linked to seems enough for what you want to
> accomplish, but it does not have the same flexibility as the Emacs'
> rgrep command does and it would take considerable additional effort if
> you want to make it more generic and usable for most people.

it doesn't take too much to implement a pratical grep in elisp with
full features such as syntax color output, filter input dir by regex,
nested dir, whether to output to dired, etc. I'm much less elisp
expert than many here, but i can do it in few days. I'd say a robust
final one can be coded in a month.

the core are already in emacs. “occur”, and “dired-do-query-replace-
regexp”, and eshell, for examples. They are just scattered and not
unified.

> Also,
> implementing it in Emacs Lisp limits the user to the supported regular
> expression style, making it impossible to use the Perl compatible one.

people can still use emacs commands that call unix utils, as they
exist.

> Just curious about the Unicode issues that you had, have you tried to
> customize the variable 'grep-command' which defaults to "grep -nH -e "
> in my setup, and pass in the "-P" parameter (for Perl regex) and then
> search for a string containing an Unicode pattern? I am usually
> grepping for English text in source code files so I did not run into
> this kind of issues.

no i haven't looked into why grepping unicode fails. It's very complex
and usually the time spent is not worth while. Everytime when you
switch computer, switch OS, get OS upgrate, telnet/remote emacs, unix
util update, using virtual box, emacs update, switching emacs distro,
the same problem occurs, and it's liketly one have to spend a day to
solve it again. On the other hand, those implemented in elisp doesn't
have this problem, such as “occur”, “eshell”, “dired-do-query-replace-
regexp”.

also, unix utils has 30 years worth of baggage. Many of the options,
the varities of grep, varieties of text pattern syntax (various regex
and file glob mechanisms), are bizarre and unwieldy and not even
flexible.

PS out of curiosity i tried you suggestion “-P” and it worked! Thanks!

is there a logical reason why that worked?

Xah

[Stefan Monnier]

You seem to have reported another problem (one Emacs can't do much
about) than the one of "grep: NUL: No such file or directory".


Stefan

[Xah Lee]

i don't see how emacs can't do anything about it.

if it found cmd.exe's find, it can simply check other files in that
dir and determine it's cmd.exe not unix. Am sure there are lots better
ways.

doesn't emach check to make sure the right param are passed to the
varieties of unix's find? why can't it do on Windows?

starting emacs with -Q and getting that error is simply a defect. If
no unix find can be found, it can simply report it so.

about the NUL, i was going to report a separate bug. But am rather
discouraged.

Xah

[Stefan Monnier]

> starting emacs with -Q and getting that error is simply a defect.
> If no unix find can be found, it can simply report it so.

I guess Emacs could report a better error, indeed, but it's hard to find
people motivated to do that. If you send us a patch we may accept
it, tho.


Stefan