Re: [sage-devel was support] Re: sage -t
Jan Groenewald
This issue is unresolved for me: users running sage -t in a system-wide
install owned by root.
On Wed, Sep 02, 2009 at 02:49:28PM +1000, Minh Nguyen wrote:
> This issue is now ticket #6861
> http://trac.sagemath.org/sage_trac/ticket/6861
# give write permissions to everyone, same as /tmp/
0 jan@muizenberg:~$ls -ld /usr/local/src/sage-4.1.1/tmp/tmp
drwxrwxrwt 2 root root 4096 2009-09-02 08:00 /usr/local/src/sage-4.1.1/tmp/tmp
# set SAGE_TESTDIR
0 jan@muizenberg:~$echo $SAGE_TESTDIR
/home/jan/.sage/
# only absolute paths work
0 jan@muizenberg:~$sage -t dpl.sage
ERROR: File ./dpl.sage is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./dpl.sage
Total time for all tests: 0.0 seconds
# still problems:
0 jan@muizenberg:~$sage -t /home/jan/dpl.sage
sage -t "/home/jan/dpl.sage"
/usr/local/src/sage-4.1.1/local/bin/sage-preparse: File file_name is missing
Traceback (most recent call last):
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 754, in <module>
test_file(argv[1], library_code = library_code)
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 587, in test_file
open(f,"w").write(s)
IOError: [Errno 13] Permission denied: './.doctest_dpl.py'
[0.1 s]
exit code: 256
----------------------------------------------------------------------
The following tests failed:
sage -t "/home/jan/dpl.sage"
Total time for all tests: 0.1 seconds
# library_code is probably false so SAGE_TESTDIR does not get set???
0 jan@muizenberg:~$grep -B8 " f = " /usr/local/src/sage-4.1.1/local/bin/sage-doctest
if library_code:
if 'SAGE_TESTDIR' not in os.environ or os.environ['SAGE_TESTDIR'] is "":
SAGE_TESTDIR = os.environ['SAGE_ROOT']+"/tmp"
else:
SAGE_TESTDIR = os.environ['SAGE_TESTDIR']
else:
SAGE_TESTDIR = '.'
f = "%s/.doctest_%s.py"%(SAGE_TESTDIR,name)
# I test it:
0 jan@muizenberg:~$sudo vim /usr/local/src/sage-4.1.1/local/bin/sage-doctest
0 jan@muizenberg:~$grep -B10 " f = " /usr/local/src/sage-4.1.1/local/bin/sage-doctest
if library_code:
if 'SAGE_TESTDIR' not in os.environ or os.environ['SAGE_TESTDIR'] is "":
SAGE_TESTDIR = os.environ['SAGE_ROOT']+"/tmp"
else:
SAGE_TESTDIR = os.environ['SAGE_TESTDIR']
else:
SAGE_TESTDIR = '.'
ftemp="/home/jan/log"
open(ftemp,"w").write("SAGE_TESTDIR="+SAGE_TESTDIR+"\n")
f = "%s/.doctest_%s.py"%(SAGE_TESTDIR,name)
# And SAGE_TESTDIR is '.'
0 jan@muizenberg:~$sage -t /home/jan/dpl.sage
sage -t "/home/jan/dpl.sage"
/usr/local/src/sage-4.1.1/local/bin/sage-preparse: File file_name is missing
Traceback (most recent call last):
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 756, in <module>
test_file(argv[1], library_code = library_code)
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 589, in test_file
open(f,"w").write(s)
IOError: [Errno 13] Permission denied: './.doctest_dpl.py'
[0.1 s]
exit code: 256
----------------------------------------------------------------------
The following tests failed:
sage -t "/home/jan/dpl.sage"
Total time for all tests: 0.1 seconds
0 jan@muizenberg:~$cat ~/log
SAGE_TESTDIR=.
# And that absolute path is a problem later, even with insane permissions:
0 jan@muizenberg:~$sudo chmod 777 /usr/local/src/sage-4.1.1/tmp/
0 jan@muizenberg:~$sudo chmod 777 /usr/local/src/sage-4.1.1/
0 jan@muizenberg:~$sage -t /home/jan/dpl.sage
sage -t "/home/jan/dpl.sage"
/usr/local/src/sage-4.1.1/local/bin/sage-preparse: File file_name is missing
File "./dpl.py", line 18
from /home/jan/dpl import *
^
SyntaxError: invalid syntax
[0.2 s]
exit code: 1024
----------------------------------------------------------------------
The following tests failed:
sage -t "/home/jan/dpl.sage"
Total time for all tests: 0.2 seconds
0 jan@muizenberg:~$
# but going back to relative path has the original problem:
0 jan@muizenberg:~$sage -t dpl.sage
ERROR: File ./dpl.sage is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./dpl.sage
Total time for all tests: 0.0 seconds
0 jan@muizenberg:~$
# same for sudo
0 jan@muizenberg:~$sudo sage -t dpl.sage
ERROR: File ./dpl.sage is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./dpl.sage
Total time for all tests: 0.0 seconds
0 jan@muizenberg:~$sudo sage -t /home/jan/dpl.sage
sage -t "/home/jan/dpl.sage"
/usr/local/src/sage-4.1.1/local/bin/sage-preparse: File file_name is missing
File "./dpl.py", line 18
from /home/jan/dpl import *
^
SyntaxError: invalid syntax
[0.2 s]
exit code: 1024
----------------------------------------------------------------------
The following tests failed:
sage -t "/home/jan/dpl.sage"
Total time for all tests: 0.2 seconds
0 jan@muizenberg:~$
I suppose this will be added to the ticket, but for now I am willing
to add write permissions to a directory where needed. Which is it?
What is my immediate solution?
regards,
Jan
-
.~.
/V\ Jan Groenewald
/( )\ www.aims.ac.za
^^-^^
Jan Groenewald
> On Wed, Sep 02, 2009 at 02:49:28PM +1000, Minh Nguyen wrote:
> > This issue is now ticket #6861
> > http://trac.sagemath.org/sage_trac/ticket/6861
On Tue, Sep 08, 2009 at 09:11:40AM +0200, Jan Groenewald wrote:
> This issue is unresolved for me: users running sage -t in a system-wide
> install owned by root.
In fact, I am not sure how sage -t is supposed top work. Where should the testfile
be? Even root cannot run it:
0 root@muizenberg:/tmp#export SAGE_TESTDIR=/tmp/
0 root@muizenberg:/tmp#sage -t dpl.sage
ERROR: File ./dpl.sage is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./dpl.sage
Total time for all tests: 0.0 seconds
0 root@muizenberg:/tmp#sage -t /tmp/dpl.sage
sage -t "/tmp/dpl.sage"
/usr/local/src/sage-4.1.1/local/bin/sage-preparse: File file_name is missing
File "./dpl.py", line 18
from /tmp/dpl import *
^
SyntaxError: invalid syntax
[0.2 s]
exit code: 1024
----------------------------------------------------------------------
The following tests failed:
sage -t "/tmp/dpl.sage"
Total time for all tests: 0.2 seconds
0 root@muizenberg:/tmp#export SAGE_TESTDIR=/root/.sage/
0 root@muizenberg:/tmp#sage -t dpl.sage
ERROR: File ./dpl.sage is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./dpl.sage
Total time for all tests: 0.0 seconds
0 root@muizenberg:/tmp#sage -t /tmp/dpl.sage
sage -t "/tmp/dpl.sage"
/usr/local/src/sage-4.1.1/local/bin/sage-preparse: File file_name is missing
File "./dpl.py", line 18
from /tmp/dpl import *
^
SyntaxError: invalid syntax
[0.2 s]
exit code: 1024
----------------------------------------------------------------------
The following tests failed:
sage -t "/tmp/dpl.sage"
Total time for all tests: 0.2 seconds
0 root@muizenberg:/tmp#
regards,
Jan
--
Minh Nguyen
On Thu, Sep 10, 2009 at 7:50 PM, Jan Groenewald <j...@aims.ac.za> wrote:
<SNIP>
> In fact, I am not sure how sage -t is supposed top work. Where should the testfile
> be? Even root cannot run it:
See ticket #6908
http://trac.sagemath.org/sage_trac/ticket/6908
for some instructions on how to doctest.
--
Regards
Minh Van Nguyen
fwc
> In fact, I am not sure how sage -t is supposed to work.
There are several problems with the file sage-doctest. Hopefully they
are resolved with the patch I posted to #6861 a few days ago.
One of them, at least, is being picked up in the error messages you
had:
> sage -t "/tmp/dpl.sage"
> /usr/local/src/sage-4.1.1/local/bin/sage-preparse: File file_name is missing
os.system('sage -preparse file_name')
which should, of course, be
os.system('sage -preparse %s' % file_name)
Francis Clarke
Department of Mathematics
Swansea University
Jan Groenewald
On Thu, Sep 10, 2009 at 08:06:19PM +1000, Minh Nguyen wrote:
> > In fact, I am not sure how sage -t is supposed top work. Where should the testfile
> > be? Even root cannot run it:
>
> See ticket #6908
> http://trac.sagemath.org/sage_trac/ticket/6908
> for some instructions on how to doctest.
Thanks.
According to that I see no way for a user to use a systemwide sage
installation to test their own modules (not intended for ever
bing included in sage, just modules they write and wish to test;
in fact part of a course on sage). It says:
"With a regular user account, the following syntax is not recommended "
and shows a couple
Our situation is a root owned installation in /usr/local/src/,
and a user using this, with their own files.sage in /home/user/,
where they wish to test some of their files.sage.
Should a user in such a case be using sage -t at all?
Or is there some other way they should be using?
I mean, I have a user (lecturer) who found it and wants
to use it to test his utility routines.
He can sage -t happily on his laptop and own installation,
but not on the computer lab desktops. There is no space
for a per-user sage installation. There is a per-desktop
root-owned installation. We set SAGE_TESTDIR to ~/.sage and
even changed permissions on /usr/local/src/sage-4.1.1/tmp/tmp,
and /usr/local/src/sage-4.1.1/tmp/, and put the test file there
so that it is in a sub directory of SAGE_ROOT.
Simon King
On Sep 10, 11:28 am, Jan Groenewald <j...@aims.ac.za> wrote:
[...]
> installation to test their own modules (not intended for ever
> bing included in sage, just modules they write and wish to test;
> in fact part of a course on sage).
and the problem was that I have no write permission to '/usr/local/
sage/tmp/tmp'.
And actually I can't understand why SAGE_TMP isn't used for temporary
data. What is the reason?
Anyway, you can use the environment variable SAGE_TESTDIR. I just did
the following on sage.math:
$ mkdir tst
$ export SAGE_TESTDIR=`pwd`/tst/
$ sage -t test.py
where test.py is some Python file in the working directory. It
worked!
So, export TEST_DIR to denote a directory in which the user has write
permission.
There is a completely different problem that once struck me: If you
want to test a file then it must be possible to attach it to a Sage
session. This is not critical for pure Python code, but it is a
problem if you have, say, a Cython file foo.pyx that has a foo.pxd
header file and also depends on external C libraries. You could not do
sage: attach foo.pyx
and you couldn't do
$ sage -t foo.pyx
either.
If this is part of your problem then I can elaborate on how I clumsily
solved that problem.
Cheers,
Simon
Simon King
> So, export TEST_DIR to denote a directory in which the user has write
> permission.
Cheers,
Simon
Minh Nguyen
On Thu, Sep 10, 2009 at 8:52 PM, Simon King <simon...@nuigalway.ie> wrote:
<SNIP>
> If this is part of your problem then I can elaborate on how I clumsily
> solved that problem.
I would love to know about the steps you took to solve the problem you
mentioned. I'm writing some documentation on how to do doctesting; see
ticket #6908
http://trac.sagemath.org/sage_trac/ticket/6908
I would like to expand that documentation with further information from you.
Jan Groenewald
On Thu, Sep 10, 2009 at 03:52:08AM -0700, Simon King wrote:
> Anyway, you can use the environment variable SAGE_TESTDIR. I just did
> the following on sage.math:
> $ mkdir tst
> $ export SAGE_TESTDIR=`pwd`/tst/
> $ sage -t test.py
> where test.py is some Python file in the working directory. It
> worked!
We did this. It works with a test file doing simple instructions, not
with doctests.
I am now trying to learn clone and hg to apply the patch at
http://trac.sagemath.org/sage_trac/attachment/ticket/6861/trac_6861_new.patch
It seems to me this is all working:
sudo -i
cd /usr/local/src/sage-4.1.1/
sage -b main # we still patch user.conf locally for less history
sage -clone sage-t-doctest-patch
cd devel/sage-sage-t-doctest-patch/
hg qinit
hg qimport /usr/local/src/trac_6861_new.patch
and then I get
0 root@muizenberg:/usr/local/src/sage-4.1.1/devel/sage-sage-t-doctest-patch#hg qpush
applying trac_6861_new.patch
unable to find 'sage-doctest' for patching
10 out of 10 hunks FAILED -- saving rejects to file sage-doctest.rej
unable to find 'sage-env' for patching
1 out of 1 hunks FAILED -- saving rejects to file sage-env.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working dir
Errors during apply, please fix and refresh trac_6861_new.patch
Simon King
On Sep 10, 12:14 pm, Jan Groenewald <j...@aims.ac.za> wrote:
> Hi
>
> On Thu, Sep 10, 2009 at 03:52:08AM -0700, Simon King wrote:
> > Anyway, you can use the environment variable SAGE_TESTDIR. I just did
> > the following on sage.math:
> > $ mkdir tst
> > $ export SAGE_TESTDIR=`pwd`/tst/
> > $ sage -t test.py
> > where test.py is some Python file in the working directory. It
> > worked!
>
> We did this. It works with a test file doing simple instructions, not
> with doctests.
If you have a file "test.py" with the following content, what happens
with "sage -t test.py"?
def foo(n):
"""
sage: foo(1)
2
sage: foo(2)
3
"""
return 2*n
I did this on sage.math, after exporting SAGE_TESTDIR to something I
can write to, and I got the expected doc test error, since of course
one gets 4, not 3, in the above doc test.
Cheers
Simon
Jan Groenewald
On Thu, Sep 10, 2009 at 04:52:48AM -0700, Simon King wrote:
> > > $ mkdir tst
> > > $ export SAGE_TESTDIR=`pwd`/tst/
> > > $ sage -t test.py
> > > where test.py is some Python file in the working directory. It
> > > worked!
> >
> > We did this. It works with a test file doing simple instructions, not
> > with doctests.
>
> What do you mean by "simple instructions" versus "doctests"?
Uh, in my earlier tests 1+1 worked but examples like below did not.
Even now, with your example:
> If you have a file "test.py" with the following content, what happens
> with "sage -t test.py"?
> def foo(n):
> """
> sage: foo(1)
> 2
> sage: foo(2)
> 3
> """
> return 2*n
0 jan@muizenberg:~$vim test.py
0 jan@muizenberg:~$mkdir tst
0 jan@muizenberg:~$mv test.py tst/
0 jan@muizenberg:~$export SAGE_TESTDIR=`pwd`/tst/
0 jan@muizenberg:~$sage -t test.py
ERROR: File ./test.py is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./test.py
Total time for all tests: 0.0 seconds
0 jan@muizenberg:~$sage -t tst/test.py
ERROR: File ./tst/test.py is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./tst/test.py
Total time for all tests: 0.0 seconds
0 jan@muizenberg:~$sage -t ~/tst/test.py
sage -t "/home/jan/tst/test.py"
Traceback (most recent call last):
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 754, in <module>
test_file(argv[1], library_code = library_code)
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 587, in test_file
open(f,"w").write(s)
IOError: [Errno 13] Permission denied: './.doctest_test.py'
[0.3 s]
exit code: 256
----------------------------------------------------------------------
The following tests failed:
sage -t "/home/jan/tst/test.py"
Total time for all tests: 0.3 seconds
> I did this on sage.math, after exporting SAGE_TESTDIR to something I
> can write to, and I got the expected doc test error, since of course
> one gets 4, not 3, in the above doc test.
This works for you?? Then I have some other problem.
Simon King
>
> <SNIP>
>
> > If this is part of your problem then I can elaborate on how I clumsily
> > solved that problem.
>
> I would love to know about the steps you took to solve the problem you
> mentioned. I'm writing some documentation on how to do doctesting; see
> ticket #6908
>
> http://trac.sagemath.org/sage_trac/ticket/6908
>
> I would like to expand that documentation with further information from you.
cohomology spkg.
Setting:
We have a Python package pGroupCohomology containing a cohomology ring
constructor class CohomologyRingFactory, an instance of that class,
and containing 5 Cython extension modules "cohomology", "resolution",
"cochain", "mtx", and "dickson". Of course, the modules contain
further classes with many methods, and also some functions.
Problem 1:
Although "from pGroupCohomology import mtx" works, one could not do
"sage -t /path/to/pGroupCohomology/mtx.pyx", for the reason mentioned
in my previous post (mtx depends on some c-library).
Solution 1:
Write a python script that recursively crawls through the package.
I.e.:
Start
* TestList = []
* ToBeTested = [pGroupCohomology]
Recursion
* import the next item FOO from ToBeTested
* append the doc string of FOO to TestList
* By introspection, find modules, classes, class instances (if FOO is
a module), methods (if FOO is a class or an instance), and append them
to ToBeTested.
Eventually, one has a long TestList, comprising the individual doc
strings from everything in pGroupCohomology. I do it in such a way
that it can be reconstructed from which item it comes.
Then, for each item from TestList, create a temporary dummy.py file
that just contains this doc string. Test it using sage -t. If an error
occurs, one can even tell the user *where* (in what module/class/
method) it occurs. Also, it is easy to determine whether a doc test is
missing, and the spkg-check script would complain if there is a method
without a doc test. I accept if a class itself has no doc test, but
all methods of a class must have a doc test.
Problem 2a):
The modules in the package import further things from Sage or from
other modules. Hence, some doc test would be done twice, or classes
from Sage (not from our package) would be doc tested.
Solution 2a):
Determine whether the item is actually defined in the module under
consideration. If it isn't, then it would not be doc tested *in this
module*. Of course, it would be tested if later the item's home module
is studied.
Problem 2b):
The classes from the package inherit from Ring, RingElement etc. By
consequence, they inherit several methods. If they aren't overloaded
then bad things may happen: One would again do a doc test for things
that are not in the package ( = wast of time), or there would be a
NotImplementedError, or the not overloaded methods have no doc tests,
resulting in a complaint of spkg-check.
Solution 2b):
I actually don't remember how, but in some way it is possible to
determine whether a method is just inherited and should thus be
disregarded.
The resulting spkg-check is part of
http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-1.1.spkg
Compare http://trac.sagemath.org/sage_trac/ticket/6750
Best regards,
Simon
Simon King
On Sep 10, 1:01 pm, Jan Groenewald <j...@aims.ac.za> wrote:
> 0 jan@muizenberg:~$vim test.py
> 0 jan@muizenberg:~$mkdir tst
> 0 jan@muizenberg:~$mv test.py tst/
> 0 jan@muizenberg:~$export SAGE_TESTDIR=`pwd`/tst/
> 0 jan@muizenberg:~$sage -t test.py
the current directory, not tst/
So, it should work if you do *not* do "mv test.py tst/"
Alternatively, it should work if you *do" "mv test.py tst/", but then
do "sage -t tst/test.py" in the last step. Of course, you need to tell
"sage -t" where the to-be-tested file sits.
Cheers,
Simon
Jan Groenewald
On Thu, Sep 10, 2009 at 05:41:47AM -0700, Simon King wrote:
> On Sep 10, 1:01 pm, Jan Groenewald <j...@aims.ac.za> wrote:
> > 0 jan@muizenberg:~$vim test.py
> > 0 jan@muizenberg:~$mkdir tst
> > 0 jan@muizenberg:~$mv test.py tst/
> > 0 jan@muizenberg:~$export SAGE_TESTDIR=`pwd`/tst/
> > 0 jan@muizenberg:~$sage -t test.py
>
> Sorry, when I said "test.py is in the working directory" then I meant
> the current directory, not tst/
> So, it should work if you do *not* do "mv test.py tst/"
0 jan@muizenberg:~$mv tst/test.py .
0 jan@muizenberg:~$export SAGE_TESTDIR=~/tst
0 jan@muizenberg:~$sage -t test.py
ERROR: File ./test.py is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./test.py
Total time for all tests: 0.0 seconds
0 jan@muizenberg:~$sage -t ~/test.py
sage -t "/home/jan/test.py"
Traceback (most recent call last):
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 754, in <module>
test_file(argv[1], library_code = library_code)
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 587, in test_file
open(f,"w").write(s)
IOError: [Errno 13] Permission denied: './.doctest_test.py'
[0.2 s]
exit code: 256
----------------------------------------------------------------------
The following tests failed:
sage -t "/home/jan/test.py"
Total time for all tests: 0.2 seconds
0 jan@muizenberg:~$
> Alternatively, it should work if you *do" "mv test.py tst/", but then
> do "sage -t tst/test.py" in the last step. Of course, you need to tell
> "sage -t" where the to-be-tested file sits.
That was shown to fail in my last post.
Simon King
On Sep 10, 1:45 pm, Jan Groenewald <j...@aims.ac.za> wrote:
[...]
> ERROR: File ./test.py is missing
> exit code: 1
> 0 jan@muizenberg:~$sage -t ~/test.py
> sage -t "/home/jan/test.py"
> Traceback (most recent call last):
> File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 754, in <module>
> test_file(argv[1], library_code = library_code)
> File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 587, in test_file
> open(f,"w").write(s)
> IOError: [Errno 13] Permission denied: './.doctest_test.py'
> [0.2 s]
> exit code: 256
write into the current directory, which is strange since you were
allowed to write test.py there.
I don't know what you did different from me. For completeness, here is
what I did on sage.math in my home directory. Of course, I have write
permissions in my home directory, but not in the default SAGE_TESTDIR.
SimonKing@sage:~$ mkdir tst
SimonKing@sage:~$ vim test.py # creating something with a failing doc
test
SimonKing@sage:~$ export SAGE_TESTDIR=~/tst
Now, the doc test fails as it is supposed to:
SimonKing@sage:~$ sage -t test.py
sage -t "test.py"
**********************************************************************
File "/home/SimonKing/test.py", line 5:
sage: foo(2)
Expected:
3
Got:
4
**********************************************************************
1 items had failures:
1 of 4 in __main__.example_0
***Test Failed*** 1 failures.
For whitespace errors, see the file ./.doctest_test.py
[1.8 s]
exit code: 1024
----------------------------------------------------------------------
The following tests failed:
Total time for all tests: 1.8 seconds
Best regards,
Simon
fwc
On Sep 10, 12:14 pm, Jan Groenewald <j...@aims.ac.za> wrote:
> I am now trying to learn clone and hg to apply the patch at http://trac.sagemath.org/sage_trac/attachment/ticket/6861/trac_6861_n...
> ...
> patch failed, rejects left in working dir
Ensure that the patch is in the directory from which you're going to
run sage, or give a full pathname below. Then do the following
(crucial that it's hg_scripts not hg_sage):
prompt$ sage
----------------------------------------------------------------------
| Sage Version 4.1.1, Release Date:
2009-08-14 |
| Type notebook() for the GUI, and license() for
information. |
----------------------------------------------------------------------
sage: hg_scripts.import_patch('trac_6861_new.patch')
There's no point in making a clone because this is the scripts
repository,
which doesn't get cloned.
I believe that my patch should solve your problem, but I admit that I
haven't checked every possibility. I'd be very interesting to hear
how it behaves in your setup.
Francis
Jan Groenewald
On Thu, Sep 10, 2009 at 05:56:16AM -0700, Simon King wrote:
> On Sep 10, 1:45 pm, Jan Groenewald <j...@aims.ac.za> wrote:
> [...]
> > 0 jan@muizenberg:~$sage -t test.py
> > ERROR: File ./test.py is missing
> > exit code: 1
>
> Strange. If test.py is in your current directory, why isn't it found?
That is what I am trying to find out. I suspect the first dot is not
the current directory, but some SAGE environment variable.
Note the absolute path below DOES find it.
> > 0 jan@muizenberg:~$sage -t ~/test.py
> > sage -t "/home/jan/test.py"
> > Traceback (most recent call last):
> > File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 754, in <module>
> > test_file(argv[1], library_code = library_code)
> > File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 587, in test_file
> > open(f,"w").write(s)
> > IOError: [Errno 13] Permission denied: './.doctest_test.py'
> > [0.2 s]
> > exit code: 256
>
> So, that's a different error. It says that you are not allowed to
> write into the current directory, which is strange since you were
> allowed to write test.py there.
Yes, once it is found the error is a new one.
Again, that current is not current. It is some SAGE dir. If I can find
it I can give users write permissions. I strace:
0 jan@muizenberg:~$strace -e open sage -t ~/test.py
open("/etc/ld.so.cache", O_RDONLY) = 3
<snip>
open("/usr/lib/locale/en_ZA.utf8/LC_CTYPE", O_RDONLY) = 3
open("/proc/meminfo", O_RDONLY) = 3
open("/proc/sys/kernel/ngroups_max", O_RDONLY) = 3
open("/usr/local/bin/sage", O_RDONLY) = 3
--- SIGCHLD (Child exited) @ 0 (0) ---
sage -t "/home/jan/test.py"
Traceback (most recent call last):
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 754, in <module>
test_file(argv[1], library_code = library_code)
File "/usr/local/src/sage-4.1.1/local/bin/sage-doctest", line 587, in test_file
open(f,"w").write(s)
IOError: [Errno 13] Permission denied: './.doctest_test.py'
[0.1 s]
exit code: 256
----------------------------------------------------------------------
The following tests failed:
sage -t "/home/jan/test.py"
Total time for all tests: 0.1 seconds
--- SIGCHLD (Child exited) @ 0 (0) ---
0 jan@muizenberg:~$
In some previous posts on sage-support I edited /usr/local/src/sage-4.1.1/local/bin/sage-doctest
to print more feedback. That library_code is never true (I think?) in our case of users
testing their own code. Or sage.math just sets some vars for you.
> I don't know what you did different from me. For completeness, here is
> what I did on sage.math in my home directory. Of course, I have write
> permissions in my home directory, but not in the default SAGE_TESTDIR.
What does
echo $SAGE<TAB>
say?
> Now, the doc test fails as it is supposed to:
I see it. I'm at a loss.
Jan Groenewald
On Thu, Sep 10, 2009 at 05:58:00AM -0700, fwc wrote:
> sage: hg_scripts.import_patch('trac_6861_new.patch')
> There's no point in making a clone because this is the scripts
> repository,
> which doesn't get cloned.
( I believe my system wide install should be owned by root, like
all other software in /usr, but there is no difference between
make and make install so I do things as root mostly. I image desktops
so I can revert mistakes )
0 jan@muizenberg:~$sudo -i
[sudo] password for jan:
0 root@muizenberg:/usr/local/src#sage
----------------------------------------------------------------------
| Sage Version 4.1.1, Release Date: 2009-08-14 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
Loading Sage library. Current Mercurial branch is: AIMS-autosave-patch
sage: hg_scripts.import_patch('trac_6861_new.patch')
cd "/usr/local/src/sage-4.1.1/local/bin" && hg status
/usr/local/src/sage-4.1.1/local/lib/python2.6/site-packages/sage/misc/hg.py:240: DeprecationWarning: os.popen3 is deprecated. Use the subprocess module.
x = os.popen3(s)
cd "/usr/local/src/sage-4.1.1/local/bin" && hg status
cd "/usr/local/src/sage-4.1.1/local/bin" && hg import "/usr/local/src/sage-4.1.1/trac_6861_new.patch"
applying /usr/local/src/sage-4.1.1/trac_6861_new.patch
sage:
Exiting SAGE (CPU time 0m0.04s, Wall time 0m21.92s).
0 root@muizenberg:/usr/local/src#
0 jan@muizenberg:~$sage -t test.py
ERROR: File ./test.py is missing
exit code: 1
----------------------------------------------------------------------
The following tests failed:
./test.py
Total time for all tests: 0.0 seconds
0 jan@muizenberg:~$sage -t ~/test.py
sage -t "/home/jan/test.py"
A mysterious error (perhaps a memory error?) occurred, which may have crashed doctest.
[0.2 s]
exit code: 768
----------------------------------------------------------------------
The following tests failed:
sage -t "/home/jan/test.py"
Total time for all tests: 0.2 seconds
0 jan@muizenberg:~$echo $SAGE_TESTDIR
/home/jan/.sage/
> I believe that my patch should solve your problem, but I admit that I
> haven't checked every possibility. I'd be very interesting to hear
> how it behaves in your setup.
I can definitely keep testing. Sometimes faster and sometimes over a few
days. Uh, I should perhaps have asked how to reverse it?
regards,
Jan
PS. the AIMS autosave patch only changes this, I believe it doesn't interfere
in the slightest:
0 jan@muizenberg:~$cat /usr/local/src/sage-4.1.1/devel/sage-AIMS-autosave-patch/sage/server/notebook/user_conf.py
"""nodoctest
"""
import conf
# patched for AIMS 1000->10 and 60->120 below
defaults = {'max_history_length':10,
'default_system':'sage',
'autosave_interval':120*60, # 2 hours in seconds
'default_pretty_print': False
}
class UserConfiguration(conf.Configuration):
def defaults(self):
return defaults
Not even worth a doctest ;)
fwc
On Sep 10, 3:47 pm, Jan Groenewald <j...@aims.ac.za> wrote:
> 0 jan@muizenberg:~$sage -t test.py
> ERROR: File ./test.py is missing
> exit code: 1
>
> ----------------------------------------------------------------------
> The following tests failed:
>
> ./test.py
> Total time for all tests: 0.0 seconds
I have permissions for the SAGE_ROOT directory. When the file exists
I get an appropriate output from the doctesting.
> 0 jan@muizenberg:~$sage -t ~/test.py
> sage -t "/home/jan/test.py"
> A mysterious error (perhaps a memory error?) occurred, which may have crashed doctest.
> [0.2 s]
> exit code: 768
>
> ----------------------------------------------------------------------
> The following tests failed:
>
> sage -t "/home/jan/test.py"
> Total time for all tests: 0.2 seconds
but the "mysterious error" message comes after doctesting has taken
place (suggesting that it has found the file). All I can suggest is
1. doing: sage -t -verbose test.py
2. looking in .sage/tmp which is what the patch sets to be
SAGE_TESTDIR
Francis
Jan Groenewald
On Thu, Sep 10, 2009 at 09:04:47AM -0700, fwc wrote:
> That's exactly what I get when ./test.py doesn't exist, whether or not
> I have permissions for the SAGE_ROOT directory. When the file exists
> I get an appropriate output from the doctesting.
In a system-wide install users do not have permissions to write
in the SAGE_ROOT directory. I have however adjusted our system
to allow write on SAGE_ROOT/tmp/tmp due to earlier errors.
sage -t should probably not require write permissions to SAGE_ROOT?
> > 0 jan@muizenberg:~$sage -t ~/test.py
> > sage -t "/home/jan/test.py"
> > A mysterious error (perhaps a memory error?) occurred, which may have crashed doctest.
> > [0.2 s]
> > exit code: 768
> >
> > ----------------------------------------------------------------------
> > The following tests failed:
> >
> > sage -t "/home/jan/test.py"
> > Total time for all tests: 0.2 seconds
>
> But this is very strange. I get something similar to the first case,
> but the "mysterious error" message comes after doctesting has taken
> place (suggesting that it has found the file). All I can suggest is
> 1. doing: sage -t -verbose test.py
> 2. looking in .sage/tmp which is what the patch sets to be
> SAGE_TESTDIR
I will have a look and report later.
regards,
Jan
fwc
1. The "mysterious error" message derives from a bug introduced by my
patch at #6861. It was triggered whenever sage -t was applied to a
full path name other than one within the devel directory. I've added
an extra patch to #6861 which deals that out, I hope. It should also
make deletion of the files in .sage/tmp work better --- when all
doctests are passed.
2. The message "ERROR: File ./test.py is missing" (produced when ./
test.py does exist) is now more of a mystery to me. It is generated
by line 99 in sage-test:
print "ERROR: File %s/%s is missing"%(os.curdir,F)
and is triggered by line 97:
if not os.path.exists(F):
(at which point F should be 'test.py'). This suggests to me that
perhaps os.path.exists isn't working as it should. It would be
interesting to try
sage: import os
sage: os.path.exists('test.py')
For me (sage-4.1.1 on Mac OS 1.6) this works as it should.
Francis
Jan Groenewald
On Thu, Sep 10, 2009 at 12:12:18PM -0700, fwc wrote:
> 1. The "mysterious error" message derives from a bug introduced by my
> patch at #6861. It was triggered whenever sage -t was applied to a
> full path name other than one within the devel directory. I've added
> an extra patch to #6861 which deals that out, I hope. It should also
> make deletion of the files in .sage/tmp work better --- when all
> doctests are passed.
Will apply soon.
> 2. The message "ERROR: File ./test.py is missing" (produced when ./
> test.py does exist) is now more of a mystery to me. It is generated
> by line 99 in sage-test:
>
> print "ERROR: File %s/%s is missing"%(os.curdir,F)
>
> and is triggered by line 97:
>
> if not os.path.exists(F):
>
> (at which point F should be 'test.py'). This suggests to me that
> perhaps os.path.exists isn't working as it should. It would be
> interesting to try
>
> sage: import os
> sage: os.path.exists('test.py')
>
> For me (sage-4.1.1 on Mac OS 1.6) this works as it should.
aha!!
0 jan@muizenberg:~$sage
----------------------------------------------------------------------
| Sage Version 4.1.1, Release Date: 2009-08-14 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
Loading Sage library. Current Mercurial branch is: AIMS-autosave-patch
sage: import os
sage: os.path.exists('test.py')
False
sage: os.path.exists('~/test.py')
False
sage: os.path.exists('/home/jan/test.py')
True
sage: os.stat('test.py')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
/home/jan/.sage/temp/muizenberg/21991/_home_jan__sage_init_sage_0.py in <module>()
OSError: [Errno 2] No such file or directory: 'test.py'
sage: pwd
'/usr/local/src/sage-4.1.1'
sage:
sage: cd
/var/autofs/misc/home/jan
sage: os.stat('test.py')
posix.stat_result(st_mode=33188, st_ino=14352965, st_dev=25L, st_nlink=1, st_uid=1000, st_gid=500, st_size=90, st_atime=1252651131, st_mtime=1252583815, st_ctime=1252586628)
xiting SAGE (CPU time 0m0.05s, Wall time 1m4.65s).
sage: os.path.exists('test.py')
True
sage:
0 jan@muizenberg:~$ipython
/var/lib/python-support/python2.6/IPython/Magic.py:38: DeprecationWarning: the sets module is deprecated
from sets import Set
Total number of aliases: 15
Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
Type "copyright", "credits" or "license" for more information.
IPython 0.9.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.
In [1]:import os
In [2]:os.path.exists('test.py')
Out[2]:True
In [3]:
Do you really want to exit ([y]/n)? y
0 jan@muizenberg:~$python -V
Python 2.6.2
0 jan@muizenberg:~$/usr/local/src/sage-4.1.1/sage-python -V
Python 2.6.2
0 jan@muizenberg:~$uname -a
Linux muizenberg 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 19:25:34 UTC 2009 x86_64 GNU/Linux
0 jan@muizenberg:~$lsb_release -d
Description: Ubuntu 9.04
Should I be setting the current working directory somehow every time
sage is started? Or is this a bug?
Jan Groenewald
\o/
> On Thu, Sep 10, 2009 at 12:12:18PM -0700, fwc wrote:
> > 1. The "mysterious error" message derives from a bug introduced by my
> > patch at #6861. It was triggered whenever sage -t was applied to a
> > full path name other than one within the devel directory. I've added
> > an extra patch to #6861 which deals that out, I hope. It should also
> > make deletion of the files in .sage/tmp work better --- when all
> > doctests are passed.
I had full-pathnames errors even before applying your first ("new") patch.
(Python import errors.)
> Will apply soon.
OK, I did, as root:
sage -b main
sage: hg_scripts.import_patch('trac_6861_new.patch')
sage: hg_scripts.import_patch('trac_6861_extra.patch')
and sage -t now works with full paths [not with relative, there
is still a problem with os.path.exists()] for the normal user
with a testfile in ~.
I then did:
sage -b AIMS-autosave-patch
(minimal changes to how many snapshots/history is kept)
and the sage -t still works, dpl.sage tests pass and test.py
shows the one intended error.
I will deploy sage, currently used in a course for 54 students, with the two
patches applied, and report any strange behaviour. Then again, students always
behave strangely.
Thanks!
fwc
On Sep 11, 7:47 am, Jan Groenewald <j...@aims.ac.za> wrote:
> Should I be setting the current working directory somehow every time
> sage is started? Or is this a bug?
mine:
trousseau-bash% pwd
/Users/mafwc
trousseau-bash% sage
----------------------------------------------------------------------
| Sage Version 4.1.1, Release Date:
2009-08-14 |
| Type notebook() for the GUI, and license() for
information. |
----------------------------------------------------------------------
sage: pwd
'/Users/mafwc'
sage:
Exiting SAGE (CPU time 0m0.07s, Wall time 0m7.96s).
trousseau-bash% cd picasso
trousseau-bash% pwd
/Users/mafwc/picasso
trousseau-bash% sage
----------------------------------------------------------------------
| Sage Version 4.1.1, Release Date:
2009-08-14 |
| Type notebook() for the GUI, and license() for
information. |
----------------------------------------------------------------------
sage: pwd
'/Users/mafwc/picasso'
I've established that during the sage startup process we first have in
line 6 of $SAGE_ROOT/sage:
CUR="`pwd`" # save the current directory, so can change back
after startup
After that the current directory changes a couple of times, then in
$SAGE_ROOT/local/bin/sage-ipython at line 21:
ipy_sage = IPython.Shell.start()
and I find that os.system('pwd') gives $SAGE_ROOT/local/bin before
this command and the directory I started sage in after. My guess is
that this is where the systems differ. I've no idea how
IPython.Shell.start() changes the current directory.
On Sep 11, 8:15 am, Jan Groenewald <j...@aims.ac.za> wrote:
> I will deploy sage, currently used in a course for 54 students, with the two
> patches applied, and report any strange behaviour. Then again, students always
> behave strangely.
Francis
Jan Groenewald
On Fri, Sep 11, 2009 at 01:31:43AM -0700, fwc wrote:
> I've established that during the sage startup process we first have in
> line 6 of $SAGE_ROOT/sage:
>
> CUR="`pwd`" # save the current directory, so can change back
> after startup
Hmmm, I had
0 jan@muizenberg:~$grep CUR /usr/local/bin/sage
CUR="/usr/local/src/sage-4.1.1/" # save the current directory, so can change back after startup
if [ "x$CUR" = "x" ]; then
export CUR
0 jan@muizenberg:~$
So for some reason CUR is set to explicitely set to what is my SAGE_ROOT
early on.
I believe that is some sed-ing that I did to set SAGE_ROOT during installs:
sed -ie 's#="....."#="/usr/local/src/sage-4.1.1/"#' /usr/local/bin/sage
Those are . which means any characte, not literal \. as I intended
somewhere in my mind, so the `pwd` which is exactly 5 characters between
apostrophes, matched it and got replaced.
Anyway, I can and did change that back to `pwd`.
My relative path problem is now fixed.
0 jan@muizenberg:~$sage
----------------------------------------------------------------------
| Sage Version 4.1.1, Release Date: 2009-08-14 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
pwd
Loading Sage library. Current Mercurial branch is: AIMS-autosave-patch
sage: pwd
'/var/autofs/misc/home/jan'
sage:
and sage -t works without full paths.
> On Sep 11, 8:15 am, Jan Groenewald <j...@aims.ac.za> wrote:
> > I will deploy sage, currently used in a course for 54 students, with the two
> > patches applied, and report any strange behaviour. Then again, students always
> > behave strangely.
>
> That should be a good doctest test!
tis done
Thanks for all the patient help. May your labour be fruitful.
mhampton
missing" means, and this post came up from google.
It might not answer Jan's question, but what does that error message
mean?
-Marshall
William Stein
>
> I was trying to figure out what "sage-preparse: File file_name is
> missing" means, and this post came up from google.
>
> It might not answer Jan's question, but what does that error message
> mean?
It's here that it is emitted:
if not os.path.exists(f):
print "%s: File %s is missing"%(sys.argv[0], f)
return
It happens when sage-preparse is called on a file that doesn't even exist.
flat:bin wstein$ sage -preparse stephen_wolfram
/Users/wstein/s/local/bin/sage-preparse: File stephen_wolfram is missing
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org