Sage 4.5.2.rc0 released
Mitesh Patel
Dan Drake and I have released Sage 4.5.2.rc0. The source is at
http://sage.math.washington.edu/home/release/sage-4.5.2.rc0/sage-4.5.2.rc0.tar
Here's an upgrade path:
http://sage.math.washington.edu/home/release/sage-4.5.2.rc0/sage-4.5.2.rc0/
The long doctests pass for us on bsd and sage.math. The t2 tests are
underway. Please build, test, and report!
This release is still in feature freeze. We don't plan to merge any
tickets unless they fix blocker issues.
== Known issues ==
A doctest failure in schemes/elliptic_curves/ell_number_field.py on
32-bit Ubuntu 9.04 x86 (Pentium 4 Prescott, gcc 4.3.3). This is a blocker:
http://trac.sagemath.org/sage_trac/ticket/9659
== Tickets ==
We merged 7 tickets in sage-4.5.2.rc0:
#8465: Minh Van Nguyen, John Palmieri: move the document "Python
Functional Programming for Mathematicians" to the classification
"Thematic Tutorials" [Reviewed by John Palmieri, Leif Leonhardy]
#9582: Carl Witty: Term order discrepancy in random test on OS X
[Reviewed by Mitesh Patel]
#9607: Rob Beezer: Doctest failure in latex.rst [Reviewed by John Palmieri]
#9608: Mitesh Patel: Docbuild warnings in sage/interfaces/matlab.py
[Reviewed by John Palmieri]
#9609: Dan Drake: Remove unnecessary files from spkg/standard [Reviewed
by John Palmieri]
#9615: Rishikesh: doctest failures with lcalc functions in 4.5.2.alpha1
[Reviewed by Dan Drake]
#9616: Mitesh Patel: Errno 16 / NFS problems with parallel/decorate.py
[Reviewed by John Palmieri]
gsw
Sage-4.5.2.rc0 does not seem to build on MacIntel OS X 10.4 for me.
The (hopefully) essential parts from the log:
...
building 'sage.libs.lcalc.lcalc_Lfunction' extension
...
g++ -L/Users/Shared/sage/sage-4.5.2.rc0/local/lib -bundle -undefined
dynamic_lookup build/temp.macosx-10.4-i386-2.6/sage/libs/lcalc/
lcalc_Lfunction.o -L/Users/Shared/sage/sage-4.5.2.rc0/local//lib -
lcsage -lm -lntl -lmpfr -lgmp -lgmpxx -lLfunction -lstdc++ -lstdc++ -
lntl -o build/lib.macosx-10.4-i386-2.6/sage/libs/lcalc/
lcalc_Lfunction.so
gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-
prototypes -I/Users/Shared/sage/sage-4.5.2.rc0/local//include -I/Users/
Shared/sage/sage-4.5.2.rc0/local//include/csage -I/Users/Shared/sage/
sage-4.5.2.rc0/devel//sage/sage/ext -I/Users/Shared/sage/
sage-4.5.2.rc0/local/include/python2.6 -c sage/libs/pari/gen.c -o
build/temp.macosx-10.4-i386-2.6/sage/libs/pari/gen.o -w
/usr/libexec/gcc/i686-apple-darwin8/4.0.1/ld: can't locate file for: -
lLfunction
collect2: ld returned 1 exit status
error: command 'g++' failed with exit status 1
sage: There was an error installing modified sage library code.
...
It seems that in the Sage library code, the usage of some "Lfunction"
library from the lcalc package was newly introduced. On my Mac, both
the (half-built) Sage-4.5.2 and the older (fuly working) Sage-4.5.1
have under local/lib/ some library "libLfunction.so". But on a Mac
under OS X (OS X 10.4 at least), this would need to be
"libLfunction.dylib" to be usable ...
How the state of matters under Cygwin (a brandnew usage of this
library might a problem there, too)?
Does anyone else see this (under OS X 10.4, or OS X 10.5, or OS X
10.6)?
Should a (blocker) ticket be opened?
Cheers,
Georg
Harald Schilly
> Dan Drake and I have released Sage 4.5.2.rc0.
Intel(R) Core(TM)2 Quad CPU Q9400
Ubuntu 8.10 32bit
gcc version 4.3.2 (Ubuntu 4.3.2-1ubuntu12)
H
gsw
for the "Lfunction" issue, the relevant tickets are #5396 resp. #4793.
It seems to be as I thought --- the lcalc spkg (#4793) never built a
correct dynamic (".dylib") library on OS X. But this was not relevant
or did break anything, until the interface between Sage and lcalc was
changed from using pexpect to using this library directly (#5396,
introduced in Sage-4.5.2.alpha1).
I didn't look at the lcalc spkg build system yet, but hopefully it is
easy and straightforward to add proper OS X support, at least for OS
10.4 (famous last words).
Again, could somebody please test the current lcalc spkg and its
interface under Cygwin?
Cheers,
Georg
davidloeffler
cores).
With SAGE_CHECK="yes", I get two failures from the Python test suite
(distutils and zlib) but all the other spkgs pass.
David
Marshall Hampton
no others), but on rerunning the tests they pass.
sage -t devel/sage-main/sage/graphs/graph.py
**********************************************************************
File "/Volumes/E/sage-4.5.2.rc0/devel/sage-main/sage/graphs/graph.py",
line 1347:
sage: cycle.order() % 2 == 0
Exception raised:
Traceback (most recent call last):
File "/Volumes/E/sage-4.5.2.rc0/local/bin/ncadoctest.py", line
1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/Volumes/E/sage-4.5.2.rc0/local/bin/sagedoctest.py", line
38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example,
filename, compileflags)
File "/Volumes/E/sage-4.5.2.rc0/local/bin/ncadoctest.py", line
1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_6[9]>", line 1, in <module>
cycle.order() % Integer(2) == Integer(0)###line 1347:
sage: cycle.order() % 2 == 0
AttributeError: 'bool' object has no attribute 'order'
**********************************************************************
File "/Volumes/E/sage-4.5.2.rc0/devel/sage-main/sage/graphs/graph.py",
line 1349:
sage: cycle.is_isomorphic(graphs.CycleGraph(cycle.order()))
Exception raised:
Traceback (most recent call last):
File "/Volumes/E/sage-4.5.2.rc0/local/bin/ncadoctest.py", line
1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/Volumes/E/sage-4.5.2.rc0/local/bin/sagedoctest.py", line
38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example,
filename, compileflags)
File "/Volumes/E/sage-4.5.2.rc0/local/bin/ncadoctest.py", line
1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_6[10]>", line 1, in <module>
cycle.is_isomorphic(graphs.CycleGraph(cycle.order()))###line
1349:
sage: cycle.is_isomorphic(graphs.CycleGraph(cycle.order()))
AttributeError: 'bool' object has no attribute 'is_isomorphic'
**********************************************************************
On Aug 1, 5:37 am, Mitesh Patel <qed...@gmail.com> wrote:
> Hello,
>
> Dan Drake and I have released Sage 4.5.2.rc0. The source is at
>
> Here's an upgrade path:
>
leif
> On a mac pro running 10.6 I got two failures in graphs/graph.py (and
> no others), but on rerunning the tests they pass.
>
> sage -t devel/sage-main/sage/graphs/graph.py
> **********************************************************************
> File "/Volumes/E/sage-4.5.2.rc0/devel/sage-main/sage/graphs/graph.py",
> line 1347:
> sage: cycle.order() % 2 == 0
> Exception raised:
> Traceback (most recent call last):
> File "/Volumes/E/sage-4.5.2.rc0/local/bin/ncadoctest.py", line
> 1231, in run_one_test
> self.run_one_example(test, example, filename, compileflags)
> File "/Volumes/E/sage-4.5.2.rc0/local/bin/sagedoctest.py", line
> 38, in run_one_example
> OrigDocTestRunner.run_one_example(self, test, example,
> filename, compileflags)
> File "/Volumes/E/sage-4.5.2.rc0/local/bin/ncadoctest.py", line
> 1172, in run_one_example
> compileflags, 1) in test.globs
> File "<doctest __main__.example_6[9]>", line 1, in <module>
> cycle.order() % Integer(2) == Integer(0)###line 1347:
> sage: cycle.order() % 2 == 0
> AttributeError: 'bool' object has no attribute 'order'
I've seen such (I don't remember if exactly there, but the same
exception/error message) on a single-core when doctesting in parallel
with too many threads.
-Leif
Mitesh Patel
> for the "Lfunction" issue, the relevant tickets are #5396 resp. #4793.
> It seems to be as I thought --- the lcalc spkg (#4793) never built a
> correct dynamic (".dylib") library on OS X. But this was not relevant
> or did break anything, until the interface between Sage and lcalc was
> changed from using pexpect to using this library directly (#5396,
> introduced in Sage-4.5.2.alpha1).
I've opened
David Joyner
> --
> You received this message because you are subscribed to the Google Groups "sage-release" group.
> To post to this group, send email to sage-r...@googlegroups.com.
> To unsubscribe from this group, send email to sage-release...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sage-release?hl=en.
>
>
Rob Beezer
make ptestlong
on 64-bit Kubuntu 9.10 on Intel Core Duo. Details:
$ echo $MAKE; echo $SAGE_PARALLEL_SPKG_BUILD
make -j2
yes
$ gcc --version
gcc (Ubuntu 4.4.1-4ubuntu8) 4.4.1
$ uname -a
Linux wave 2.6.31-16-generic #53-Ubuntu SMP Tue Dec 8 04:02:15 UTC
2009 x86_64 GNU/Linux
Alex Ghitza
Successful upgrade from sage-4.5, and all tests pass with make ptestlong
on:
[aghitza@soleil sage-4.5]$ uname -a
Linux soleil.ms.unimelb.edu.au 2.6.18-194.8.1.el5 #1 SMP Wed Jun 23 10:52:51 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
Best,
Alex
--
Alex Ghitza -- http://aghitza.org/
Lecturer in Mathematics -- The University of Melbourne -- Australia
John Cremona
John
Dr. David Kirkby
> There's a patch for #9659 up for review.
>
> John
I've give it positive review
Dave
r Rishikesh
it built fine. It was on 10.5. So 10.5 should not be a problem. I do
not have access to a 10.4 machine.
Rishi
leif
> I am trying to replicate the problem. The last time, I built on OSX,
> it built fine. It was on 10.5. So 10.5 should not be a problem. I do
> not have access to a 10.4 machine.
According to the release announcement here, rc0 has built fine and
passed all (long) tests on bsd.math, which is a MacOS X 10.4.0 box.
-Leif
kcrisman
> According to the release announcement here, rc0 has built fine and
> passed all (long) tests on bsd.math, which is a MacOS X 10.4.0 box.
>
version - very annoying, I agree, but true. I think that's Darwin
8.
I'll see if I can confirm this problem on my own 10.4 (Tiger) box, but
gsw's explanation makes perfect sense.
- kcrisman
leif
>> According to the release announcement here, rc0 has built fine and
>> passed all (long) tests on bsd.math, which is a MacOS X 10.4.0 box.
>>
>
> No, that is Darwin 10.4.0. Mac OS X 10.4 is a different Darwin
> version - very annoying, I agree, but true. I think that's Darwin
> 8.
Oh. Sorry.
> I'll see if I can confirm this problem on my own 10.4 (Tiger) box, but
> gsw's explanation makes perfect sense.
I thought that, too, but wondered why it worked on bsd.math (and other
OS X boxes). I do not deal with apples though.
-Leif
John H Palmieri
> Hello,
>
> Dan Drake and I have released Sage 4.5.2.rc0. The source is at
>
> Here's an upgrade path:
>
> The long doctests pass for us on bsd and sage.math. The t2 tests are
> underway. Please build, test, and report!
t2.math: still in progress because I can't figure out where to build
it: /scratch doesn't have enough room. I'm trying in my home
directory, which is far from ideal.
Mac OS X 10.6, 64-bit: all tests passed
skynet machines cicero, cleo, eno, flavius, lena, menas, sextus,
taurus: all tests passed.
skynet machine iras: two non-repeatable failures: graphs/genus.pyx (I
reported the same problem for the alpha releases) and schemes/
elliptic_curves/BSD.py.
skynet machine mark (solaris on sparc): after using the cvxopt spkg
from #9657, one failure: a timeout on ssmod.py. Given the slowness of
this machine, it's not a surprise that it has a timeout failure. If we
get #9657 merged, this machine will then build Sage successfully "out
of the box". This is progress.
skynet machine fulvia (solaris on x86, 32-bit): R fails to build. If
I then touch spkg/installed/r-..., everything else builds, and there
are about 10 files with doctest failures. Not a blocker for this
release (but perhaps for Sage 5.0?) since this platform is not yet
officially supported. We need to working on it, though: see #9026 and
related tickets.
fulvia (64-bit): lots of problems here, as discussed at #9026. Again,
not a blocker.
--
John
William Stein
> On Aug 1, 3:37 am, Mitesh Patel <qed...@gmail.com> wrote:
>> Hello,
>>
>> Dan Drake and I have released Sage 4.5.2.rc0. The source is at
>>
>> http://sage.math.washington.edu/home/release/sage-4.5.2.rc0/sage-4.5....
>>
>> Here's an upgrade path:
>>
>> http://sage.math.washington.edu/home/release/sage-4.5.2.rc0/sage-4.5....
>>
>> The long doctests pass for us on bsd and sage.math. The t2 tests are
>> underway. Please build, test, and report!
>
> sage.math: all tests passed
>
> t2.math: still in progress because I can't figure out where to build
> it: /scratch doesn't have enough room. I'm trying in my home
> directory, which is far from ideal.
I ordered a new fast internal disk for t2 today (with David Kirkby's help).
>
> Mac OS X 10.6, 64-bit: all tests passed
>
> skynet machines cicero, cleo, eno, flavius, lena, menas, sextus,
> taurus: all tests passed.
>
> skynet machine iras: two non-repeatable failures: graphs/genus.pyx (I
> reported the same problem for the alpha releases) and schemes/
> elliptic_curves/BSD.py.
>
> skynet machine mark (solaris on sparc): after using the cvxopt spkg
> from #9657, one failure: a timeout on ssmod.py. Given the slowness of
> this machine, it's not a surprise that it has a timeout failure. If we
> get #9657 merged, this machine will then build Sage successfully "out
> of the box". This is progress.
>
> skynet machine fulvia (solaris on x86, 32-bit): R fails to build. If
> I then touch spkg/installed/r-..., everything else builds, and there
> are about 10 files with doctest failures. Not a blocker for this
> release (but perhaps for Sage 5.0?) since this platform is not yet
> officially supported. We need to working on it, though: see #9026 and
> related tickets.
>
> fulvia (64-bit): lots of problems here, as discussed at #9026. Again,
> not a blocker.
>
> --
> John
>
> --
> You received this message because you are subscribed to the Google Groups "sage-release" group.
> To post to this group, send email to sage-r...@googlegroups.com.
> To unsubscribe from this group, send email to sage-release...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sage-release?hl=en.
>
>
--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org
kcrisman
things like this when building certain Sage library pyx/c files
(upgrading to rc0):
/usr/libexec/gcc/powerpc-apple-darwin8/4.0.1/ld: warning can't open
dynamic library: libpari-gmp.dylib referenced from: /Users/.../
sage-4.5.1/local/lib/libcsage.dylib (checking for undefined symbols
may be affected) (No such file or directory, errno = 2)
I am naively asking whether this is actually related - I suppose
probably not, since there is no mention of .so files. I do appear to
have this file in local/bin, but maybe something isn't working because
of this so business?
- kcrisman
Justin C. Walker
On Aug 1, 2010, at 03:37 , Mitesh Patel wrote:
> Hello,
>
> Dan Drake and I have released Sage 4.5.2.rc0. The source is at
>
> http://sage.math.washington.edu/home/release/sage-4.5.2.rc0/sage-4.5.2.rc0.tar
Built from scratch on Mac OS X, 10.5.8 (dual quad Xeon). No problems
and all tests passed. 10.6 build to follow.
--
Justin C. Walker, Curmudgeon at Large
Institute for the Absorption of Federal Funds
-----------
My wife 'n kids 'n dogs are gone,
I can't get Jesus on the phone,
But Ol' Milwaukee's Best is my best friend.
-----------
gsw
On 4 Aug., 04:56, kcrisman <kcris...@gmail.com> wrote:
> Just as a followup, on my OS X 10.4 Tiger machine, I get lots of
> things like this when building certain Sage library pyx/c files
> (upgrading to rc0):
>
> /usr/libexec/gcc/powerpc-apple-darwin8/4.0.1/ld: warning can't open
> dynamic library: libpari-gmp.dylib referenced from: /Users/.../
> sage-4.5.1/local/lib/libcsage.dylib (checking for undefined symbols
> may be affected) (No such file or directory, errno = 2)
>
> I am naively asking whether this is actually related - I suppose
> probably not, since there is no mention of .so files. I do appear to
> have this file in local/bin, but maybe something isn't working because
> of this so business?
>
> - kcrisman
That's been the case for a long time now, I always see loads of that
in my build/install logs for the OS X 10.4 Sage. There does not seem
to be any problem on OS X 10.5 or later, only on OS X 10.4 Tiger. On
the latter platform, I consider it to be an acceptable nuisance --- to
my knowledge, no one else noticed, or ever reported this (up to now),
or did complain about it.
Personally, I think it is totally unrelated to "this so business", but
rather has to do with the "rpath" feature of statically building into
binaries and libraries "pointers" to other dynamic libraries on which
they depend, and generally how this information is evaluated. There
are subtle issues, maybe the fix for Tiger would be just to change
nothing else but the order of "libraries to include" in some specific
place in the build system.
Cheers,
Georg
leif
FWIW, I did see exactly the same two doctest errors (and also only
these) when running "make ptestlong" (with only the default number of
threads, which is 2 on a [single-core] Pentium 4 with HT) the first time
on 4.5.2 (final), system otherwise idling.
I've *never* seen these before with just 2 threads on that machine
(running Ubuntu 9.04 x86); they didn't show up in a second run.
Dr. David Kirkby
> On a mac pro running 10.6 I got two failures in graphs/graph.py (and
> no others), but on rerunning the tests they pass.
I note Leif says today:
========================================
I've *never* seen these before with just 2 threads on that machine
(running Ubuntu 9.04 x86); they didn't show up in a second run.
=========================================
(Both comments were on sage-release, but seem more appropite to put on sage-devel).
Why are so many tests failing, but then passing on reruns?
This is happening far too often, where tests fail, but then pass. I've seen it
on Linux, OS X and Solaris, with several different tests.
I've seen it on my Blade 2000 which has ony 2 CPUs but 8 GB RAM. Theres no way
that should be a resource shortage. (I often run tests on my Blade 1000, which
has a lot less RAM, so I can attribute the odd failure to a resource shortage).
I think the tickets that aim to improve the doctesting framework should be given
fairly high priority in merging them. (I did not write one single one of them!!!)
Seriously though, if we can't trust the doctests, what can we trust?
Another possibility is that the code may be buggy, so pass some times and fail
others. It might be worth running any tests which fail then pass a large number
of times.
Nathann Cohen
subgraph_search is totally deterministic, but this "cycle" object is
created from a random graph. When there is no cycle in the graph, the
value returned in None or False, which in any case has no order
attribute. To fix it : just make it a larger Random Graph, or change
the doctest. I'll be doing one of these in #9715
Oh yeah. And I have to add that the probability that this happens,
though I did not compute it is.... small...
{{{
sage: n = 100000
sage: sum( Graph(graphs.RandomBipartite(10, 10, .5)).is_forest() for i
in xrange(n))/n
0
}}}
I will add something to check this does not come from a mistake is
subgraph_search, which I would hate.
Nathann
On Aug 10, 3:53 am, "Dr. David Kirkby" <david.kir...@onetel.net>
wrote:
Dr. David Kirkby
> Oh. Well, I was at first very worried because the code from
> subgraph_search is totally deterministic, but this "cycle" object is
> created from a random graph. When there is no cycle in the graph, the
> value returned in None or False, which in any case has no order
> attribute. To fix it : just make it a larger Random Graph, or change
> the doctest. I'll be doing one of these in #9715
>
> Oh yeah. And I have to add that the probability that this happens,
> though I did not compute it is.... small...
>
> {{{
> sage: n = 100000
> sage: sum( Graph(graphs.RandomBipartite(10, 10, .5)).is_forest() for i
> in xrange(n))/n
> 0
> }}}
>
> I will add something to check this does not come from a mistake is
> subgraph_search, which I would hate.
>
> Nathann
This gets back to the point I made to you before about thinking about how things
might fail - not fixing bugs when someone reports a failure.
But I think the problem extends well beyond a few lines you may have wrote in
one bit of code. I've seen this so many times now, with code written by
different people. BSD.py has often been a cause of failures for me, which later
pass.
I created a wiki page where people can list doctests which fail then pass.
http://wiki.sagemath.org/doctests-that-fail-then-pass
Hopefully some culprits can be identified, and those looked at in more detail in
case they have code which like yours has a small probability of failing under
some cases.
I think part of the problem may be resource shortages, which can result in
timeouts. Unfortunately, some timeouts have been reported as "missing files",
though I think that bug has been squashed now.
I know someome remarked one test was taking over 1 GB RAM on his system and when
I tested it, it was over 500 MB. Those sort of tests, could easily cause
problems on systems with little RAM, especially if they happen to be run in
parallel with another test that uses a lot of RAM.
Dave
Nathann Cohen
> things might fail - not fixing bugs when someone reports a failure.
Not really. I knew this docstring could fail, it is just very unikely.
Take 1000 random numbers between 0 and 10, and check their mean is
between 4 and 6. Yeah, of course it can fail. You can take a long list
of "0". The very same happened for this docstring. When you are taking
a random graph, there is a non-zero probability that it has no
edges... It is just... unlikely :-D
> But I think the problem extends well beyond a few lines you may have wrote
> in one bit of code. I've seen this so many times now, with code written by
> different people. BSD.py has often been a cause of failures for me, which
> later pass.
>
> I created a wiki page where people can list doctests which fail then pass.
>
> http://wiki.sagemath.org/doctests-that-fail-then-pass
Of course, any doctest which does not pass should be immediately
fixed. We can consider ourselves lucky when the actualy mistake is in
the docstring, and not the methods themselves :-)
> Hopefully some culprits can be identified and those looked at in more
> detail in case they have code which like yours has a small probability of
> failing under some cases.
<.<
>.>
> I think part of the problem may be resource shortages, which can result in
> timeouts. Unfortunately, some timeouts have been reported as "missing
> files", though I think that bug has been squashed now.
Not in this case. That code requires almost no ressources. This
failure just means that everybody can hope to win at the lottery :-D
> I know someome remarked one test was taking over 1 GB RAM on his system and
> when I tested it, it was over 500 MB. Those sort of tests, could easily
> cause problems on systems with little RAM, especially if they happen to be
> run in parallel with another test that uses a lot of RAM.
LP sometimes requires things like that... I think the worst docstring
on this regard in the Graph/LP library is Graph.is_minor, and it is
not ~so~ bad....
Nathann
leif
>> Dr. David Kirkby wrote:
>> This gets back to the point I made to you before about thinking about
>> how things might fail - not fixing bugs when someone reports a failure.
>
> Not really. I knew this docstring could fail, it is just very unikely.
> Take 1000 random numbers between 0 and 10, and check their mean is
> between 4 and 6. Yeah, of course it can fail. You can take a long list
> of "0". The very same happened for this docstring. When you are taking
> a random graph, there is a non-zero probability that it has no
> edges... It is just... unlikely :-D
That's why we use *pseudo* random numbers. Such doctests should be the
same across different systems, and in every run on the same machine.
(It's not a bad idea to add a doctest ensuring this, i.e. comparing the
graph to the one expected.)
>
>> But I think the problem extends well beyond a few lines you may have wrote
>> in one bit of code. I've seen this so many times now, with code written by
>> different people. BSD.py has often been a cause of failures for me, which
>> later pass.
>>
>> I created a wiki page where people can list doctests which fail then pass.
>>
>> http://wiki.sagemath.org/doctests-that-fail-then-pass
Fill it! ;-)
>
> Of course, any doctest which does not pass should be immediately
> fixed. We can consider ourselves lucky when the actualy mistake is in
> the docstring, and not the methods themselves :-)
Some doctest fixes do just the opposite; they fix the symptom rather
than (try to track down and fix) the cause.
Though there have been IMHO needless changes to the actual code rather
than just the doctest, too, e.g. adding sorted(...) around returned
values in the tested function itself, rather than just sorting prior to
comparison in the doctest.
>
>> Hopefully some culprits can be identified and those looked at in more
>> detail in case they have code which like yours has a small probability of
>> failing under some cases.
No doctest should have a small probability of failing; except for e.g.
memory addresses or pids given in examples (not really tests), all
doctests should be deterministic (see above wrt. random numbers).
>
> <.<
>
>> .>
>
>> I think part of the problem may be resource shortages, which can result in
>> timeouts. Unfortunately, some timeouts have been reported as "missing
>> files", though I think that bug has been squashed now.
I think the odd "file not found" is finally past. (Btw, the logs are
made to allow investigation of the real causes.)
>
> Not in this case. That code requires almost no ressources. This
> failure just means that everybody can hope to win at the lottery :-D
>
>> I know someome remarked one test was taking over 1 GB RAM on his system and
>> when I tested it, it was over 500 MB. Those sort of tests, could easily
>> cause problems on systems with little RAM, especially if they happen to be
>> run in parallel with another test that uses a lot of RAM.
I've successfully built and tested Sage on a system (also running a GUI)
with just 768MB RAM (without ever swapping); watching the memory
consumption during building and testing, I'm quite sure half of that
would suffice. (Requirements of course rise when building/testing in
parallel, but I also didn't observe excessive memory usage with 6 or 16
threads on other machines.)
[Not testing return codes of critical functions is of course another
issue, and one potential cause of such failures. Despite having (badly
documented) exceptions, I wish Python had something like
"-Wunused-result" (and "-Werror")...]
I don't remember any of such irreproducible doctest failures being
reported from sequential testing, so to me these smell more of
non-reentrant code, or broken synchronization that's meant to avoid race
conditions.
-Leif
Dr. David Kirkby
> Nathann Cohen wrote:
>>> Dr. David Kirkby wrote:
>>> This gets back to the point I made to you before about thinking about
>>> how things might fail - not fixing bugs when someone reports a failure.
>>
>> Not really. I knew this docstring could fail, it is just very unikely.
>> Take 1000 random numbers between 0 and 10, and check their mean is
>> between 4 and 6. Yeah, of course it can fail. You can take a long list
>> of "0". The very same happened for this docstring. When you are taking
>> a random graph, there is a non-zero probability that it has no
>> edges... It is just... unlikely :-D
>
> That's why we use *pseudo* random numbers. Such doctests should be the
> same across different systems, and in every run on the same machine.
> (It's not a bad idea to add a doctest ensuring this, i.e. comparing the
> graph to the one expected.)
>
>
>>
>>> But I think the problem extends well beyond a few lines you may have wrote
>>> in one bit of code. I've seen this so many times now, with code written by
>>> different people. BSD.py has often been a cause of failures for me, which
>>> later pass.
>>>
>>> I created a wiki page where people can list doctests which fail then pass.
>>>
>>> http://wiki.sagemath.org/doctests-that-fail-then-pass
>
> Fill it! ;-)
You were the one that reported the failure earlier today!
> I've successfully built and tested Sage on a system (also running a GUI)
> with just 768MB RAM (without ever swapping); watching the memory
> consumption during building and testing, I'm quite sure half of that
> would suffice. (Requirements of course rise when building/testing in
> parallel, but I also didn't observe excessive memory usage with 6 or 16
> threads on other machines.)
Jeroen Demeyer reported some of the doctests related to elliptic curves used a
lot of RAM (> 1 GB). When I asked for a particular test, he said:
sage/schemes/elliptic_curves/heegner.py
When I run the doctest, it used just under 700 MB for me on a 32-bit Solaris
SPARC build. (I'm guessing it might use more on a 64-bit build, depending on how
the data is stored).
I believe BSD.py is related to elliptic curves - that test has failed for me
more than once, then passed.
> I don't remember any of such irreproducible doctest failures being
> reported from sequential testing, so to me these smell more of
> non-reentrant code, or broken synchronization that's meant to avoid race
> conditions.
>
>
> -Leif
Yes, I think sequential testing is more reliable - just a lot slower on
multi-core machines.
Dave
leif
> On 08/10/10 06:39 PM, leif wrote:
>> Nathann Cohen wrote:
>>> Dr. David Kirkby wrote:
>>>> But I think the problem extends well beyond a few lines you may have
>>>> wrote
>>>> in one bit of code. I've seen this so many times now, with code
>>>> written by
>>>> different people. BSD.py has often been a cause of failures for me,
>>>> which
>>>> later pass.
>>>>
>>>> I created a wiki page where people can list doctests which fail then
>>>> pass.
>>>>
>>>> http://wiki.sagemath.org/doctests-that-fail-then-pass
>>
>> Fill it! ;-)
>
> You were the one that reported the failure earlier today!
Not really, I just "confirmed" a doctest error previously reported by
Marshall Hampton, for rc0. ;-)
>> I don't remember any of such irreproducible doctest failures being
>> reported from sequential testing, so to me these smell more of
>> non-reentrant code, or broken synchronization that's meant to avoid race
>> conditions.
>
> Yes, I think sequential testing is more reliable - just a lot slower on
> multi-core machines.
:-) I didn't want to say "don't test in parallel", but name a possible
reason that should be examined, since running "normal" code in parallel
(or even running Sage on heavy-loaded machines) might give similar
errors, caused by the same thing.
-Leif
Dr. David Kirkby
>>> I don't remember any of such irreproducible doctest failures being
>>> reported from sequential testing, so to me these smell more of
>>> non-reentrant code, or broken synchronization that's meant to avoid race
>>> conditions.
>>
>> Yes, I think sequential testing is more reliable - just a lot slower on
>> multi-core machines.
>
> :-) I didn't want to say "don't test in parallel", but name a possible
> reason that should be examined, since running "normal" code in parallel
> (or even running Sage on heavy-loaded machines) might give similar
> errors, caused by the same thing.
>
>
> -Leif
I do not know enough about the doc testing framework to really comment. I'd like
to see the times of tests recorded, so one could correlated that with RAM usage,
CPU usage, system error messages.
I know there are problems on the *.math network due to the fact the ZIL log has
been disabled on disk.math. Some of the failures might be due to that, as I see
error messages on t2 that relate to NFS errors. Being about to correlate times
of failures with system log messages would be useful.
If a doc test is taking over 1 GB on one machine, but you can doctest sage in
768 MB of RAM without swapping, then something is strange.
I don't really trust the doctests much. I don't however to claim to know what's
wrong.