Anybody tried search_* in 4.1.alpha1?

[kcrisman]

In trying to find out what was going on in another post, I tried e.g.

sage: search_src("cannot;")
find: -exec: no terminating ";"

sage: search_doc("cannot")
find: -exec: no terminating ";"

Trying to add ; at various points didn't help. Is this just me
(4.1.alpha1 fresh build on G4 X.4)? Is GAP running in the
background? But when I quit, there is no message about that, and in
any case presumably GAP doesn't do the codebase search.

- kcrisman

[William Stein]

On Fri, Jun 26, 2009 at 2:26 PM, kcrisman<kcri...@gmail.com> wrote:
>
> In trying to find out what was going on in another post, I tried e.g.
>
> sage: search_src("cannot;")
> find: -exec: no terminating ";"

Maybe you don't have grep installed? What happens if you do

sage -grep cannot

from the command line?

>
> sage: search_doc("cannot")
> find: -exec: no terminating ";"
>
> Trying to add ; at various points didn't help.  Is this just me
> (4.1.alpha1 fresh build on G4 X.4)?  Is GAP running in the
> background?  But when I quit, there is no message about that, and in
> any case presumably GAP doesn't do the codebase search.
>
> - kcrisman
> >
>

--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

[kcrisman]

On Jun 26, 8:47 am, William Stein <wst...@gmail.com> wrote:

> On Fri, Jun 26, 2009 at 2:26 PM, kcrisman<kcris...@gmail.com> wrote:
>
> > In trying to find out what was going on in another post, I tried e.g.
>
> > sage: search_src("cannot;")
> > find: -exec: no terminating ";"
>
> Maybe you don't have grep installed?  What happens if you do
>
>    sage -grep cannot
>
> from the command line?

That works great.

[Nick Alexander]

>
> Trying to add ; at various points didn't help. Is this just me
> (4.1.alpha1 fresh build on G4 X.4)? Is GAP running in the
> background? But when I quit, there is no message about that, and in
> any case presumably GAP doesn't do the codebase search.

Why do you think GAP is relevant? The relevant command seems to be

cmd = 'sage -grepdoc "%s" | grep "%s"'%(s,extra)

(The default is extra=''). So can you try 'sage -grepdoc' on your
architecture?

Nick

[kcrisman]

On Jun 26, 11:28 am, Nick Alexander <ncalexan...@gmail.com> wrote:
> > Trying to add ; at various points didn't help.  Is this just me
> > (4.1.alpha1 fresh build on G4 X.4)?  Is GAP running in the
> > background?  But when I quit, there is no message about that, and in
> > any case presumably GAP doesn't do the codebase search.
>
> Why do you think GAP is relevant? The relevant command seems to be

Only because of the ;

>
>      cmd = 'sage -grepdoc "%s" | grep "%s"'%(s,extra)
>
> (The default is extra='').  So can you try 'sage -grepdoc' on your  
> architecture?
>

Similarly to sage -grep, works like a charm. This is the first Sage
installation where this problem has come up, so the problem must have
come in since 4.0.2. Where is the code for these sage -foo commands?
I found them once but have forgotten where they are.

The following from man find might be relevant, since that is the error
message which I am getting:

-exec utility [argument ...];
True if the program named utility returns a zero value as
its
exit status. Optional arguments may be passed to the
utility.
The expression must be terminated by a semicolon
(``;''). If the
string ``{}'' appears anywhere in the utility name or the
argu-
ments it is replaced by the pathname of the current file.
Utility will be executed from the directory from which
find was
executed. Utility and arguments are not subject to the
further
expansion of shell patterns and constructs.

but I don't know where find is used here. Does grep call find? (That
is obviously a shell newbie question, apologies.)

- kcrisman

[Nick Alexander]

> Where is the code for these sage -foo commands?
> I found them once but have forgotten where they are.

SAGE_ROOT/local/bin/sage-foo

> but I don't know where find is used here. Does grep call find?

Yes, depending on what parameters you give it. The string you give to
search_doc is actually considered a shell command first and a regular
expression second. Can you try "cannot\;" and "cannot\\;" (note
slashes)? I am a little curious that just "cannot" fails at all.

Nick

[kcrisman]

On Jun 26, 12:06 pm, Nick Alexander <ncalexan...@gmail.com> wrote:
> > Where is the code for these sage -foo commands?
> > I found them once but have forgotten where they are.
>
> SAGE_ROOT/local/bin/sage-foo

Thanks.

> > but I don't know where find is used here.  Does grep call find?
>
> Yes, depending on what parameters you give it.  The string you give to  
> search_doc is actually considered a shell command first and a regular  
> expression second.  Can you try "cannot\;" and "cannot\\;" (note  
> slashes)?  I am a little curious that just "cannot" fails at all.

I get the same result no matter what I put in. Even

sage: search_src("p-adics")
find: -exec: no terminating ";"

though in 4.0.1 that gets a page of hits.

I'm really sorry this is so weird. This is OSX.4 on a PPC, so perhaps
something got changed in those scripts recently?

- kcrisman

[John H Palmieri]

Yes, it got changed in #5806. Can you try

find SAGE_ROOT/devel/sage/sage// \( -name *.py \) -exec grep -i -H
cannot {} +

from the command line, replacing SAGE_ROOT with the appropriate path?
Also, what does 'which find' tell you? (Does anyone know whether the
'find' command changed from OS X 10.4 to 10.5?)

John

[kcrisman]

> > I'm really sorry this is so weird.  This is OSX.4 on a PPC, so perhaps
> > something got changed in those scripts recently?
>

> Yes, it got changed in #5806. Can you try
>
>  find SAGE_ROOT/devel/sage/sage// \( -name *.py \) -exec grep -i -H
> cannot {} +
>

Same error. I tried putting a ; in various places, but to no avail.
Reading # 5806 discussion seems to indicate perhaps something is
different for X.4 or for PPC?

> from the command line, replacing SAGE_ROOT with the appropriate path?
> Also, what does 'which find' tell you?  (Does anyone know whether the
> 'find' command changed from OS X 10.4 to 10.5?)

/usr/bin/find

- kcrisman

[John H Palmieri]

On Jun 26, 10:33 am, kcrisman <kcris...@gmail.com> wrote:
> > > I'm really sorry this is so weird.  This is OSX.4 on a PPC, so perhaps
> > > something got changed in those scripts recently?
>
> > Yes, it got changed in #5806. Can you try
>
> >  find SAGE_ROOT/devel/sage/sage// \( -name *.py \) -exec grep -i -H
> > cannot {} +
>
> Same error.  I tried putting a ; in various places, but to no avail.

How about if you replace the "+" at the end with "\;"?

As I said on #5806, I really hate the find command...

John

[kcrisman]

On Jun 26, 2:24 pm, John H Palmieri <jhpalmier...@gmail.com> wrote:
> On Jun 26, 10:33 am, kcrisman <kcris...@gmail.com> wrote:
>
> > > > I'm really sorry this is so weird.  This is OSX.4 on a PPC, so perhaps
> > > > something got changed in those scripts recently?
>
> > > Yes, it got changed in #5806. Can you try
>
> > >  find SAGE_ROOT/devel/sage/sage// \( -name *.py \) -exec grep -i -H
> > > cannot {} +
>
> > Same error.  I tried putting a ; in various places, but to no avail.
>
> How about if you replace the "+" at the end with "\;"?

Bingo!

find Desktop/sage-4.1.alpha1/devel/sage/sage// \( -name *.py \) -exec
grep -i -H Palmieri {} \;

Desktop/sage-4.1.alpha1/devel/sage/sage///algebras/
steenrod_algebra.py:- John H. Palmieri (2008-07-30): version 0.9
Desktop/sage-4.1.alpha1/devel/sage/sage///algebras/
steenrod_algebra.py:# Copyright (C) 2008 John H. Palmieri
<palm...@math.washington.edu>
...
Desktop/sage-4.1.alpha1/devel/sage/sage///misc/latex_macros.py:- John
H. Palmieri (2009-03)
Desktop/sage-4.1.alpha1/devel/sage/sage///misc/sagedoc.py:- J. H.
Palmieri (2009-04-11): fix for #5754 plus doctests

[Arnaud Bergeron]

2009/6/26 kcrisman <kcri...@gmail.com>:

>
> On Jun 26, 2:24 pm, John H Palmieri <jhpalmier...@gmail.com> wrote:
>> On Jun 26, 10:33 am, kcrisman <kcris...@gmail.com> wrote:
>>
>> > > > I'm really sorry this is so weird.  This is OSX.4 on a PPC, so perhaps
>> > > > something got changed in those scripts recently?
>>
>> > > Yes, it got changed in #5806. Can you try
>>
>> > >  find SAGE_ROOT/devel/sage/sage// \( -name *.py \) -exec grep -i -H
>> > > cannot {} +
>>
>> > Same error.  I tried putting a ; in various places, but to no avail.
>>
>> How about if you replace the "+" at the end with "\;"?

The '+' thing is a GNUism. That's why it doesn't work on 10.4. You should use

find $SAGE_ROOT/devel/sage/sage/ \( -name '*.py' \) -print0 | xargs -0
grep -i -H 'cannot'

So that you get the behavior that '+' gives (limit the number of
invocations of grep) and stay portable.

> Bingo!
>
> find Desktop/sage-4.1.alpha1/devel/sage/sage// \( -name *.py \) -exec
> grep -i -H Palmieri {} \;
>
> Desktop/sage-4.1.alpha1/devel/sage/sage///algebras/
> steenrod_algebra.py:- John H. Palmieri (2008-07-30): version 0.9
> Desktop/sage-4.1.alpha1/devel/sage/sage///algebras/
> steenrod_algebra.py:#       Copyright (C) 2008 John H. Palmieri
> <palm...@math.washington.edu>
> ...
> Desktop/sage-4.1.alpha1/devel/sage/sage///misc/latex_macros.py:- John
> H. Palmieri (2009-03)
> Desktop/sage-4.1.alpha1/devel/sage/sage///misc/sagedoc.py:- J. H.
> Palmieri (2009-04-11): fix for #5754 plus doctests
>
> >
>

--
La brigade SnW veut vous recruter - http://brigade.snw.googlepages.com

[gsw]

On 26 Jun., 21:34, Arnaud Bergeron <aberge...@gmail.com> wrote:
> 2009/6/26 kcrisman <kcris...@gmail.com>:

>
>
>
> > On Jun 26, 2:24 pm, John H Palmieri <jhpalmier...@gmail.com> wrote:
> >> On Jun 26, 10:33 am, kcrisman <kcris...@gmail.com> wrote:
>
> >> > > > I'm really sorry this is so weird.  This is OSX.4 on a PPC, so perhaps
> >> > > > something got changed in those scripts recently?
>
> >> > > Yes, it got changed in #5806. Can you try
>
> >> > >  find SAGE_ROOT/devel/sage/sage// \( -name *.py \) -exec grep -i -H
> >> > > cannot {} +
>
> >> > Same error.  I tried putting a ; in various places, but to no avail.
>
> >> How about if you replace the "+" at the end with "\;"?
>
> The '+' thing is a GNUism.  That's why it doesn't work on 10.4.  You should use

Wow,
on my MacIntel OS X 10.4.11 box the following command *does work
fine*:

find /Users/Shared/sage/sage-4.0.2/devel/sage/sage// \( -name *.py \) -

exec grep -i -H cannot {} +

especially this command has got this "GNUish +" at the end, but on my
MacPPC OS X 10.4.11 box I get this strange error message (find: -exec:
no terminating ";"), too.
Weird!
So it is not a OS X 10.4.11 issue, but rather a MacPPC versus MacIntel
issue --- OS X 10.5 won't install on my MacPPC box (it has got only
550MHz), could someone please check this on a MacPPC with OS X 10.5,
e.g. at the Sage build farm?

>
> find $SAGE_ROOT/devel/sage/sage/ \( -name '*.py' \) -print0 | xargs -0
> grep -i -H 'cannot'
>

This version of the command works fine on *both* the OS X 10.4.11
MacIntel and MacPPC, and is fast.

> So that you get the behavior that '+' gives (limit the number of
> invocations of grep) and stay portable.
>
>
>
> > Bingo!
>
> > find Desktop/sage-4.1.alpha1/devel/sage/sage// \( -name *.py \) -exec
> > grep -i -H Palmieri {} \;
>

This one works on both boxes, too, but behaves way slower.

Cheers,
gsw

> > Desktop/sage-4.1.alpha1/devel/sage/sage///algebras/
> > steenrod_algebra.py:- John H. Palmieri (2008-07-30): version 0.9
> > Desktop/sage-4.1.alpha1/devel/sage/sage///algebras/
> > steenrod_algebra.py:#       Copyright (C) 2008 John H. Palmieri

> > <palmi...@math.washington.edu>

[John H Palmieri]

On Jun 26, 1:15 pm, gsw <GeorgSWe...@googlemail.com> wrote:
> On 26 Jun., 21:34, Arnaud Bergeron <aberge...@gmail.com> wrote:
>
>
>
>
>
> > 2009/6/26 kcrisman <kcris...@gmail.com>:
>
> > > On Jun 26, 2:24 pm, John H Palmieri <jhpalmier...@gmail.com> wrote:
> > >> On Jun 26, 10:33 am, kcrisman <kcris...@gmail.com> wrote:
>
> > >> > > > I'm really sorry this is so weird.  This is OSX.4 on a PPC, so perhaps
> > >> > > > something got changed in those scripts recently?
>
> > >> > > Yes, it got changed in #5806. Can you try
>
> > >> > >  find SAGE_ROOT/devel/sage/sage// \( -name *.py \) -exec grep -i -H
> > >> > > cannot {} +
>
> > >> > Same error.  I tried putting a ; in various places, but to no avail.
>
> > >> How about if you replace the "+" at the end with "\;"?
>
> > The '+' thing is a GNUism.  That's why it doesn't work on 10.4.  You should use

It's also in Mac OS X 10.5, which still claims to be BSD. See the web
page

<http://developer.apple.com/documentation/Darwin/Reference/ManPages/
man1/find.1.html>

For portability and ease of debugging (but not necessarily for speed),
I'm working on redoing all of this in python, using os.walk to replace
the find command. Assuming that this is a good idea, I have an
implementation question: at the moment, if you do

sage: search_src('submatrix', 'starting', '0')

then you get matches like

matrix/matrix0.pyx: Get The 2 x 2 submatrix of M, starting at
row index and column

Notice that the third argument '0' is not in the Sage source code, but
only in the file name. This is not what the documentation says that
the function is supposed to do. Should we change the documentation or
the behavior?

That is, we have

def search_src(string, extra1='', extra2='', extra3='', extra4='',
extra5='', interact=True):

Should the strings extra1, etc., be searched for only in the source
code, or should they be searched for both in the source code and the
file name?

John

[John H Palmieri]

My vote, by the way, is for the first of these: only search in the
source code. As it currently stands, the following two commands
behave differently:

sage: search_src('submatrix', 'starting', '0')

sage: search_src('0', 'submatrix', 'starting')

because the first argument is not searched for in file names, while
the remaining arguments are.

John

[John Cremona]

2009/6/26 John H Palmieri <jhpalm...@gmail.com>:

It had never occurred to me that the search strings would be looked
for in filenames, only in lines of code!

John

>
>  John
>
> >
>

[Craig Citro]

> Should the strings extra1, etc., be searched for only in the source
> code, or should they be searched for both in the source code and the
> file name?
>

I've definitely used the "search path too" behavior on purpose at
various times. In particular, doing things like adding "matrix" to the
search essentially narrows the result to the sage/matrix/ directory,
which can be handy. Of course, this is clearly a bit of a hack, and
inconsistent (as John is pointing out). So how about instead, we
implement what I'm actually trying to do with this -- add a "path"
optional argument to the search_* functions. (If you're using os.walk,
this should be pretty trivial to add, I think.) It'd be especially fun
if you could give it the name of a module (so that I could tab
complete), too.

-cc

[John H Palmieri]

Good ideas. There's a patch up at

<http://trac.sagemath.org/sage_trac/ticket/6429>

Please try it out, especially if you have access to a PPC OS X 10.4
machine. It seems to work on sage.math and on my Intel OS X 10.5 box.

John