sage-4.5.alpha4 released

44 views
Skip to first unread message

Robert Miller

unread,
Jul 6, 2010, 8:30:37 AM7/6/10
to sage-release
http://sage.math.washington.edu/home/release/sage-4.5.alpha4/sage-4.5.alpha4.tar

This is a short set of additions to alpha3, and I was going to start
merging patches to sage-main before releasing, but because of

http://trac.sagemath.org/sage_trac/ticket/9435

I wanted to release this separately, since there are changes to deps,
etc. Sorry about the rapid-fire alphas...

SPKG changes are now frozen. Only blocker spkg tickets will be merged
from now until sage-4.5 final. The next release will be rc0.

--
Robert L. Miller
http://www.rlmiller.org/

William Stein

unread,
Jul 6, 2010, 8:46:02 AM7/6/10
to sage-r...@googlegroups.com
On Tue, Jul 6, 2010 at 2:30 PM, Robert Miller <r...@rlmiller.org> wrote:
> http://sage.math.washington.edu/home/release/sage-4.5.alpha4/sage-4.5.alpha4.tar
>
> This is a short set of additions to alpha3, and I was going to start
> merging patches to sage-main before releasing, but because of
>
> http://trac.sagemath.org/sage_trac/ticket/9435
>
> I wanted to release this separately, since there are changes to deps,
> etc. Sorry about the rapid-fire alphas...

Thanks. Even sage-4.5.alpha3 failed with lots of doctest errors for
me on sage.math.
I'll try this alpha4 and if it passes all tests on sage.math, then
I'll try testing on skynet,
etc.

William

>
> SPKG changes are now frozen. Only blocker spkg tickets will be merged
> from now until sage-4.5 final. The next release will be rc0.
>
> --
> Robert L. Miller
> http://www.rlmiller.org/
>

> --
> You received this message because you are subscribed to the Google Groups "sage-release" group.
> To post to this group, send email to sage-r...@googlegroups.com.
> To unsubscribe from this group, send email to sage-release...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sage-release?hl=en.
>
>

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

leif

unread,
Jul 6, 2010, 10:28:00 AM7/6/10
to sage-r...@googlegroups.com

We should perhaps make #9314 ("LaTeX representation of negative symbolic
fractions still broken") a blocker for 4.5.
http://trac.sagemath.org/sage_trac/ticket/9314


-Leif

William Stein

unread,
Jul 6, 2010, 12:36:17 PM7/6/10
to sage-r...@googlegroups.com
Hi,

Has anybody successfully got all tests to pass for sage-4.5.alpha[3-4]
on sage.math.washington.edu? For me, many, many maxima-related tests
fail.

-- William

John H Palmieri

unread,
Jul 6, 2010, 1:22:01 PM7/6/10
to sage-release
On Jul 6, 9:36 am, William Stein <wst...@gmail.com> wrote:
> Hi,
>
> Has anybody successfully got all tests to pass for sage-4.5.alpha[3-4]
> on sage.math.washington.edu?  For me, many, many maxima-related tests
> fail.

Yes, both built for me. For alpha3, the only failure was for mip.pyx
(fixed by #9432). For alpha4, I got a failure for tests/startup.py,
which seems to happen for me pretty regularly when I test in parallel,
but that was the only one.

These were builds from scratch with SAGE_PARALLEL_SPKG_BUILD='yes':
took half an hour, not counting building the docs. I haven't tried
updating, and I haven't tried a serial build. Maybe I'll do that
next.

--
John

Justin C. Walker

unread,
Jul 6, 2010, 2:21:15 PM7/6/10
to sage-r...@googlegroups.com

On Jul 6, 2010, at 09:36 , William Stein wrote:

> Hi,
>
> Has anybody successfully got all tests to pass for sage-4.5.alpha[3-4]
> on sage.math.washington.edu? For me, many, many maxima-related tests
> fail.

Stock 4.5.a3 built w/o problems for me, and only the known 'mip.py'
tests failed. This is on 10.5.8 and 10.6.4.

Justin

--
Justin C. Walker, Curmudgeon at Large
Institute for the Absorption of Federal Funds
-----------
My wife 'n kids 'n dogs are gone,
I can't get Jesus on the phone,
But Ol' Milwaukee's Best is my best friend.
-----------


Robert Miller

unread,
Jul 6, 2010, 5:36:36 PM7/6/10
to sage-r...@googlegroups.com
My build results for sage-4.5.alpha4:

* sage.math with "make" : all tests passed except the startup time test [1]

* bsd.math with "make" : all tests passed!

* geom.math with "make" : all tests passed!
* geom.math with MAKE="make -j10" : all tests passed!

[1] I tried this six times, and it failed exactly three times. The
machine was over half loaded when I did this:

sage -t -long devel/sage-main/sage/tests/startup.py
**********************************************************************
File "/mnt/usb1/scratch/rlm/sage-4.5.alpha4/devel/sage-main/sage/tests/startup.py",
line 6:
sage: if os.uname()[1] == 'sage.math.washington.edu':
print float(os.popen("sage -startuptime>/dev/null; sage
-startuptime|grep
sage.all").readlines()[Integer(0)].split()[Integer(1)]) <
RealNumber('1.5')
else: print True # nothing when not on sage.math
Expected:
True
Got:
False
**********************************************************************

Based on these very positive results, I am announcing a FULL FEATURE FREEZE.

I would just like to emphasize that sage-4.5 is way cooler than sliced
bread, and nearly as cool as the Dutch football team: it includes GLPK
as a standard spkg, so for example, one can compute whether a graph is
hamiltonian without downloading anything extra. It also includes an
awesome new parallel building capability: David Kirkby has built Sage
from scratch in under five minutes! And, having just watched the end
of the Netherlands-Uruguay world cup game during Sage Days 23 at
Leiden, I've decided to give this one a title:

== Sage-4.5: De Nederlandse Editie ==

...Think of the bits as bright orange if you like.

I'd like to suggest that we give this until the end of the week to see
what crops up before we release. I'd also like to suggest that David
Loeffler should do a very short 4.6 release, since he has already done
much of the work necessary to put together a release which only merges
patches to sage-main (e.g.). I feel bad since there were so many good
tickets which I didn't merge, but I had a few targets in mind, such as
all the spkg tickets, GLPK, intra/inter spkg parallel building, etc.
etc.

The only remaining issues that I know of are the failures that William
had in testing, but since I haven't been able to reproduce them, I'm
curious if he can.

While I'm rambling about this release, I should give extra special
thanks to Nathann Cohen, David Kirkby, David Loeffler, Willem Jan
Palenstijn and Justin Walker for all their help in getting this
release together.

Nathann Cohen

unread,
Jul 6, 2010, 5:57:17 PM7/6/10
to sage-r...@googlegroups.com
Hail Robert ! :-)

Nathann

John H Palmieri

unread,
Jul 6, 2010, 6:44:20 PM7/6/10
to sage-release
On Jul 6, 2:36 pm, Robert Miller <r...@rlmiller.org> wrote:
> My build results for sage-4.5.alpha4:
>
> * sage.math with "make" : all tests passed except the startup time test [1]

I got "all tests passed" on both sage.math and on a mac running OS X
10.6. I've built it successfully on t2.math and am currently running
doctests there. If there are any failures, I'll report them.

Both html and pdf versions of the docs build, with two warnings for
the reference manual:

/Applications/sage_builds/sage-4.5.alpha4/devel/sage/doc/en/reference/
graphs.rst:4: (WARNING/2) toctree references unknown document u'sage/
graphs/pq_trees.py'
/Applications/sage_builds/sage-4.5.alpha4/local/lib/python2.6/site-
packages/sage/schemes/elliptic_curves/sha_tate.py:docstring of
sage.schemes.elliptic_curves.sha_tate.Sha.bound_kato:12: (WARNING/2)
Definition list ends without a blank line; unexpected unindent.

See <http://trac.sagemath.org/sage_trac/ticket/9442> for fixes.

--
John

David Kirkby

unread,
Jul 6, 2010, 6:58:37 PM7/6/10
to sage-release
On Jul 6, 10:36 pm, Robert Miller <r...@rlmiller.org> wrote:
> My build results for sage-4.5.alpha4:

I'm just building on 't2'. It still takes ages on there, even with
parallel builds, due to ATLAS. I think I might install ATLAS globally
on that machine, then us use the environment variable to specify the
location of the ATLAS libraries, rather than build them.

> I would just like to emphasize that sage-4.5 is way cooler than sliced
> bread, and nearly as cool as the Dutch football team: it includes GLPK
> as a standard spkg, so for example, one can compute whether a graph is
> hamiltonian without downloading anything extra. It also includes an
> awesome new parallel building capability: David Kirkby has built Sage
> from scratch in under five minutes! And, having just watched the end
> of the Netherlands-Uruguay world cup game during Sage Days 23 at
> Leiden, I've decided to give this one a title:
>
> == Sage-4.5: De Nederlandse Editie ==

That's only the Sage library I can build in under 5 minutes - not the
complete Sage code!! It will need some fast computer to build the
whole of Sage in 5 minutes!

It only just beats the 5 minute goal post at 4m59.635s. I doubt that
is a record - I expect sage.math could build the library quicker,
though I've not tried it.

It also needs

http://trac.sagemath.org/sage_trac/ticket/9399
and
http://trac.sagemath.org/sage_trac/ticket/9097

merged into the library to get it to build.

> While I'm rambling about this release, I should give extra special
> thanks to Nathann Cohen, David Kirkby, David Loeffler, Willem Jan
> Palenstijn and Justin Walker for all their help in getting this
> release together.
>
> --
> Robert L. Millerhttp://www.rlmiller.org/

Any chance of

http://trac.sagemath.org/sage_trac/ticket/9399
and
http://trac.sagemath.org/sage_trac/ticket/9097

getting merged into 4.5 for my help? They are bug fixes to the
library, not new features or .spkg updates.

Dave

John H Palmieri

unread,
Jul 6, 2010, 7:28:33 PM7/6/10
to sage-release
On Jul 6, 3:58 pm, David Kirkby <david.kir...@onetel.net> wrote:
> On Jul 6, 10:36 pm, Robert Miller <r...@rlmiller.org> wrote:
>
> > My build results for sage-4.5.alpha4:
>
> I'm just building on 't2'. It still takes ages on there, even with
> parallel builds, due to ATLAS. I think I might install ATLAS globally
> on that machine, then us use the environment variable to specify the
> location of the ATLAS libraries, rather than build them.

Personally, I just manually copy over the appropriate files from
include/ and lib/ from a previous Sage build of ATLAS, and I touch the
file spkg/installed/atlas-3.8.3.p12. In fact, I have the files saved
in /home/palmieri/t2/from-atlas-3.8.3.p12/ on sage.math (and t2.math)
if you want to use them. Note that the environment variable approach
might not work on Solaris until #9356 is merged, depending on what
library files you actually build.

--
John

Dr. David Kirkby

unread,
Jul 6, 2010, 7:43:33 PM7/6/10
to sage-r...@googlegroups.com
On 07/ 7/10 12:28 AM, John H Palmieri wrote:
> On Jul 6, 3:58 pm, David Kirkby<david.kir...@onetel.net> wrote:
>> On Jul 6, 10:36 pm, Robert Miller<r...@rlmiller.org> wrote:
>>
>>> My build results for sage-4.5.alpha4:
>>
>> I'm just building on 't2'. It still takes ages on there, even with
>> parallel builds, due to ATLAS. I think I might install ATLAS globally
>> on that machine, then us use the environment variable to specify the
>> location of the ATLAS libraries, rather than build them.
>
> Personally, I just manually copy over the appropriate files from
> include/ and lib/ from a previous Sage build of ATLAS, and I touch the
> file spkg/installed/atlas-3.8.3.p12. In fact, I have the files saved
> in /home/palmieri/t2/from-atlas-3.8.3.p12/ on sage.math (and t2.math)
> if you want to use them.

Thank you John.

ATLAS is building now, and has been for some hours. I'm just about to go to bed
and by the time I wake it will have built. So there's not a lot of point in me
changing things now.

But in future I will either install these globally or use yours.

> Note that the environment variable approach
> might not work on Solaris until #9356 is merged, depending on what
> library files you actually build.

That's what I intended using. I will have to wait for that then. But for this
build, I'm just going to let ATLAS complete.

> --
> John
>


Dave

John H Palmieri

unread,
Jul 6, 2010, 9:59:31 PM7/6/10
to sage-release
On Jul 6, 3:44 pm, John H Palmieri <jhpalmier...@gmail.com> wrote:
> On Jul 6, 2:36 pm, Robert Miller <r...@rlmiller.org> wrote:
>
> > My build results for sage-4.5.alpha4:
>
> > * sage.math with "make" : all tests passed except the startup time test [1]
>
> I got "all tests passed" on both sage.math and on a mac running OS X
> 10.6.  I've built it successfully on t2.math and am currently running
> doctests there.  If there are any failures, I'll report them.

On t2.math, one failure:


sage -t -long devel/sage/sage/misc/trace.py
**********************************************************************
File "/home/palmieri/t2/sage-4.5.alpha4/devel/sage-main/sage/misc/
trace.py", line 54:
sage: _ = s.expect('100')
Exception raised:
Traceback (most recent call last):
File "/home/palmieri/t2/sage-4.5.alpha4/local/bin/
ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/home/palmieri/t2/sage-4.5.alpha4/local/bin/
sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example,
filename, compileflags)
File "/home/palmieri/t2/sage-4.5.alpha4/local/bin/
ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_1[6]>", line 1, in <module>
_ = s.expect('100')###line 54:
sage: _ = s.expect('100')
File "/home/palmieri/t2/sage-4.5.alpha4/local/lib/python/site-
packages/pexpect.py", line 912, in expect
return self.expect_list(compiled_pattern_list, timeout,
searchwindowsize)
File "/home/palmieri/t2/sage-4.5.alpha4/local/lib/python/site-
packages/pexpect.py", line 989, in expect_list
raise TIMEOUT (str(e) + '\n' + str(self))
TIMEOUT: Timeout exceeded in read_nonblocking().
<pexpect.spawn instance at 0x2c005d0>
version: 2.0 ($Revision: 1.151 $)
command: /home/palmieri/t2/sage-4.5.alpha4/sage
args: ['/home/palmieri/t2/sage-4.5.alpha4/sage']
patterns:
100
buffer (last 100 chars):
before (last 100 chars): *

**********************************************************************
c

after: <class 'pexpect.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: 0
pid: 22383
child_fd: 4
timeout: 30
delimiter: <class 'pexpect.EOF'>
logfile: None
maxread: 2000
searchwindowsize: None
delaybeforesend: 0.1
**********************************************************************
File "/home/palmieri/t2/sage-4.5.alpha4/devel/sage-main/sage/misc/
trace.py", line 61:
sage: print s.before[s.before.find('-'):]
Expected:
---...
ipdb> c
2 * 5
Got:

----------------------------------------------------------------------
| Sage Version 4.5.alpha4, Release Date:
2010-07-06 |
| Type notebook() for the GUI, and license() for
information. |

----------------------------------------------------------------------

**********************************************************************

* *
* Warning: this is a prerelease version, and it may be
unstable. *

* *

**********************************************************************
c
<BLANKLINE>
**********************************************************************
1 items had failures:
2 of 10 in __main__.example_1
***Test Failed*** 2 failures.

--
John

John H Palmieri

unread,
Jul 6, 2010, 10:08:29 PM7/6/10
to sage-release
On Jul 6, 6:59 pm, John H Palmieri <jhpalmier...@gmail.com> wrote:
> On Jul 6, 3:44 pm, John H Palmieri <jhpalmier...@gmail.com> wrote:
>
> > On Jul 6, 2:36 pm, Robert Miller <r...@rlmiller.org> wrote:
>
> > > My build results for sage-4.5.alpha4:
>
> > > * sage.math with "make" : all tests passed except the startup time test [1]
>
> > I got "all tests passed" on both sage.math and on a mac running OS X
> > 10.6.  I've built it successfully on t2.math and am currently running
> > doctests there.  If there are any failures, I'll report them.
>
> On t2.math, one failure:

There were also some timeouts, but no other failures. The build is
in /home/palmieri/t2/sage-4.5.alpha4/ (on t2.math) if anyone wants to
take a look.

--
John

William Stein

unread,
Jul 7, 2010, 12:08:51 AM7/7/10
to sage-r...@googlegroups.com
On Tue, Jul 6, 2010 at 11:36 PM, Robert Miller <r...@rlmiller.org> wrote:
> The only remaining issues that I know of are the failures that William
> had in testing, but since I haven't been able to reproduce them, I'm
> curious if he can.

When I build sage-4.5.alpha4 (and sage-4.5.alpha3) on sage.math, it
builds, but fails a lot of maxima-related tests.

http://sage.math.washington.edu/home/wstein/build/sage-4.5.alpha4/

(See testlong.log).

I built Sage using this script:

http://sage.math.washington.edu/home/wstein/buildbot

This sets SAGE_FAT_BINARY="yes", since that's what we do for all
binaries we distribute.
It's possible that this is necessary in order to see this problem.

-- William

Pat LeSmithe

unread,
Jul 7, 2010, 12:09:57 AM7/7/10
to sage-r...@googlegroups.com
On 07/06/2010 11:36 AM, William Stein wrote:
> Has anybody successfully got all tests to pass for sage-4.5.alpha[3-4]
> on sage.math.washington.edu? For me, many, many maxima-related tests
> fail.

I have seen all tests pass with 4.5.alpha{1,3,4}, but I've also noticed
many Maxima-related failures on occasion on sage.math with
SAGE_PARALLEL_SPKG_BUILD="yes" and 4, 6, or 12 parallel jobs. Two
representative errors are mentioned at

http://trac.sagemath.org/sage_trac/ticket/9274#comment:17

It seems that reinstalling the Maxima spkg via 'sage -f' "fixes" the
problems. So does renaming SAGE_ROOT.

Another tidbit:

$ ./sage


----------------------------------------------------------------------
| Sage Version 4.5.alpha4, Release Date: 2010-07-06 |

...
**********************************************************************
sage: solve(x,x)


Traceback (most recent call last):

...
TypeError: error evaluating "load(to_poly_solver)":
Error executing code in Maxima
CODE:
load(to_poly_solver);
Maxima ERROR:

Could not find `to_poly_solver' using paths in
file_search_maxima,file_search_lisp.
-- an error. To debug this try: debugmode(true);
sage: solve(x,x)
[x == 0]
sage:

But repetition does not fix

sage: QQ[2^(1/3)]


Traceback (most recent call last):

...
ValueError: variable names must be alphanumeric, but one is '2^(1/3)'
which is not.

for me.


William Stein

unread,
Jul 7, 2010, 12:12:15 AM7/7/10
to sage-r...@googlegroups.com
On Wed, Jul 7, 2010 at 6:09 AM, Pat LeSmithe <qed...@gmail.com> wrote:
> On 07/06/2010 11:36 AM, William Stein wrote:
>> Has anybody successfully got all tests to pass for sage-4.5.alpha[3-4]
>> on sage.math.washington.edu?  For me, many, many maxima-related tests
>> fail.
>
> I have seen all tests pass with 4.5.alpha{1,3,4}, but I've also noticed
> many Maxima-related failures on occasion on sage.math with
> SAGE_PARALLEL_SPKG_BUILD="yes" and 4, 6, or 12 parallel jobs.  Two
> representative errors are mentioned at
>
> http://trac.sagemath.org/sage_trac/ticket/9274#comment:17

Yes, that's the problem I'm seeing.

Note that I am *not* building with SAGE_PARALLEL_SPKG_BUILD set...

William Stein

unread,
Jul 7, 2010, 12:22:00 AM7/7/10
to sage-r...@googlegroups.com
On Wed, Jul 7, 2010 at 6:12 AM, William Stein <wst...@gmail.com> wrote:
> On Wed, Jul 7, 2010 at 6:09 AM, Pat LeSmithe <qed...@gmail.com> wrote:
>> On 07/06/2010 11:36 AM, William Stein wrote:
>>> Has anybody successfully got all tests to pass for sage-4.5.alpha[3-4]
>>> on sage.math.washington.edu?  For me, many, many maxima-related tests
>>> fail.
>>
>> I have seen all tests pass with 4.5.alpha{1,3,4}, but I've also noticed
>> many Maxima-related failures on occasion on sage.math with
>> SAGE_PARALLEL_SPKG_BUILD="yes" and 4, 6, or 12 parallel jobs.  Two
>> representative errors are mentioned at
>>
>> http://trac.sagemath.org/sage_trac/ticket/9274#comment:17
>
> Yes, that's the problem I'm seeing.
>
> Note that I am *not* building with SAGE_PARALLEL_SPKG_BUILD set...

Rebuilding Maxima with "sage -f" does fix the problem. And I did
keep SAGE_FAT_BINARY set to "yes".
So this is a tricky issue, indeed. But it absolutely has to be
resolved before releasing sage-4.5.

-- William

Justin C. Walker

unread,
Jul 7, 2010, 1:05:16 AM7/7/10
to sage-r...@googlegroups.com

On Jul 6, 2010, at 05:30 , Robert Miller wrote:

> http://sage.math.washington.edu/home/release/sage-4.5.alpha4/sage-4.5.alpha4.tar
>
> This is a short set of additions to alpha3, and I was going to start
> merging patches to sage-main before releasing, but because of
>
> http://trac.sagemath.org/sage_trac/ticket/9435
>
> I wanted to release this separately, since there are changes to deps,
> etc. Sorry about the rapid-fire alphas...

This just in:

Mac OS X, 10.5.8, Dual Quad Xeon: built w/o problems, one test failure:
sage -t "devel/sage/sage/libs/galrep/wrapper.pyx"
(two failures in galrep.GalRep().non_surjective_primes())
Total time for all tests: 7726.5 seconds

The full log file for the test failure is at:
sage.math.washington.edu:~justin/4.5.a4.log

Mac OS X, 10.6.4, Core i7: built w/o problems, no test failures.

Justin

--
Justin C. Walker, Curmudgeon-At-Large


Institute for the Absorption of Federal Funds
--------

If you're not confused,
You're not paying attention
--------

Robert Miller

unread,
Jul 7, 2010, 1:14:07 AM7/7/10
to sage-release
--- William,

1. Are you still seeing the issue at

http://trac.sagemath.org/sage_trac/ticket/9435

in alpha4, or can this ticket be closed?

2. Can you open a ticket about the maxima failures? Please make it a blocker.

It seems that we need either SAGE_FAT_BINARY or
SAGE_PARALLEL_SPKG_BUILD set to "yes" in order to see this... And does
running sage-location (perhaps with a spoofed
local/lib/sage-current-location.txt to force it) fix it in each case?

I'm afraid this is outside my area of expertise. Expert opinions welcome.

> But it absolutely has to be resolved before releasing sage-4.5

Indeed!

--- John,

Can you open a ticket for the t2 failure you saw, and make it a blocker?

--- Justin,

Same, regarding the galrep failure you saw?

--- David,

I was discussing the 64-bit build fixes with Willem Jan Palenstijn
yesterday, and it seems as if adding "-m64" to CFLAGS etc. does not
work on all platforms. I am thinking specifically in terms of #9097. I
will post later with more details, since this ticket is on my short
list...

Rob Beezer

unread,
Jul 7, 2010, 1:43:36 AM7/7/10
to sage-release
All tests pass with

make ptestlong

on 64-bit KUbuntu 9.10 on Intel Core Duo.

Nice.

Rob

Justin C. Walker

unread,
Jul 7, 2010, 1:44:01 AM7/7/10
to sage-r...@googlegroups.com

On Jul 6, 2010, at 22:14 , Robert Miller wrote:

> --- Justin,
>
> Same, regarding the galrep failure you saw?

Done: #9445.

John H Palmieri

unread,
Jul 7, 2010, 2:02:26 AM7/7/10
to sage-release
On Jul 6, 10:14 pm, Robert Miller <r...@rlmiller.org> wrote:
>
> --- John,
>
> Can you open a ticket for the t2 failure you saw, and make it a blocker?

It's #9446.

--
John

Dr. David Kirkby

unread,
Jul 7, 2010, 5:43:34 AM7/7/10
to sage-r...@googlegroups.com
On 07/ 7/10 02:59 AM, John H Palmieri wrote:
> On Jul 6, 3:44 pm, John H Palmieri<jhpalmier...@gmail.com> wrote:
>> On Jul 6, 2:36 pm, Robert Miller<r...@rlmiller.org> wrote:
>>
>>> My build results for sage-4.5.alpha4:
>>
>>> * sage.math with "make" : all tests passed except the startup time test [1]
>>
>> I got "all tests passed" on both sage.math and on a mac running OS X
>> 10.6. I've built it successfully on t2.math and am currently running
>> doctests there. If there are any failures, I'll report them.
>
> On t2.math, one failure:
>
>
> sage -t -long devel/sage/sage/misc/trace.py
> **********************************************************************
> File "/home/palmieri/t2/sage-4.5.alpha4/devel/sage-main/sage/misc/
> trace.py", line 54:
> sage: _ = s.expect('100')
> Exception raised:
> Traceback (most recent call last):
> File "/home/palmieri/t2/sage-4.5.alpha4/local/bin/
> ncadoctest.py", line 1231, in run_one_test

This is odd, as this test passes for me on 't2'.

I've put further info on the ticket you created at

http://trac.sagemath.org/sage_trac/ticket/9446

kirkby@t2:[/tmp/kirkby/sage-4.5.alpha4] $ ./sage -t -long
devel/sage/sage/misc/trace.py
sage -t -long "devel/sage/sage/misc/trace.py"
[29.4 s]

----------------------------------------------------------------------
All tests passed!
Total time for all tests: 29.4 seconds

Note
* I did build Sage in parallel
* I have not run all the tests, so don't know if I get any other failures.

One possible reason for your failure is that the configuration of the ZFS pools
on 'disk.math' have been modified (not by me I would add) to increase NFS
performance by disabling a logging device - a practice considered unsafe.

Looking in the logs on 't2' I see the following - clearly showing problems in
the subdirectory /home/palmieri/t2/sage-4.5.alpha3, but NOT not one containing
alpha4. Has any of the files from the alpha3 build been copied to the alpha4?


Jul 6 12:06:06 t2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server:
disk][Mntpt: /home]NFS op OP_CLOSE got error NFS4ERR_STALE causing recovery
action NR_STALE.
Jul 6 12:06:06 t2 nfs: [ID 286389 kern.info] NOTICE: [NFS4][Server:
disk][Mntpt: /home]File ./palmieri/t2/sage-4.5.alpha3/local/bin/python2.6
(rnode_pt: 3003cad4018) was closed due to NFS recovery error on server
disk(failed to recover from NFS4ERR_STALE NFS4ERR_STALE)
Jul 6 12:06:06 t2 nfs: [ID 941083 kern.info] NOTICE: NFS4 FACT SHEET:
Jul 6 12:06:06 t2 Action: NR_STALE
Jul 6 12:06:06 t2 NFS4 error: NFS4ERR_STALE
Jul 6 13:25:28 t2 nfs: [ID 236337 kern.info] NOTICE: [NFS4][Server:
disk][Mntpt: /home]NFS op OP_CLOSE got error NFS4ERR_STALE causing recovery
action NR_STALE.
Jul 6 13:25:28 t2 nfs: [ID 286389 kern.info] NOTICE: [NFS4][Server:
disk][Mntpt: /home]File
./palmieri/t2/sage-4.5.alpha3/local/lib/gap-4.4.12/bin/gap.sh (rnode_pt:
6004da64c50) was closed due to NFS recovery error on server disk(failed to
recover from NFS4ERR_STALE NFS4ERR_STALE)
Jul 6 13:25:28 t2 nfs: [ID 941083 kern.info] NOTICE: NFS4 FACT SHEET:

I personally always suspect NFS issues whenever I see non-reproducible problems
on *.math.washington.edu. But in this case, the only failure recorded is in the
alpha3 directory. I would add, I'm not sure if the NFS client, (t2) would
necessarily log such a problem, so its possible it occurred when you built Sage,
but it was not logged.

I'll run the full test suite and let you know of any failures.

Dave

Dr. David Kirkby

unread,
Jul 7, 2010, 8:36:37 AM7/7/10
to sage-r...@googlegroups.com
On 07/ 7/10 02:59 AM, John H Palmieri wrote:
> On Jul 6, 3:44 pm, John H Palmieri<jhpalmier...@gmail.com> wrote:
>> On Jul 6, 2:36 pm, Robert Miller<r...@rlmiller.org> wrote:
>>
>>> My build results for sage-4.5.alpha4:
>>
>>> * sage.math with "make" : all tests passed except the startup time test [1]
>>
>> I got "all tests passed" on both sage.math and on a mac running OS X
>> 10.6. I've built it successfully on t2.math and am currently running
>> doctests there. If there are any failures, I'll report them.
>
> On t2.math, one failure:
>
>
> sage -t -long devel/sage/sage/misc/trace.py

Well, I get a 5 failures on 't2'.

The following tests failed:

sage -t -long devel/sage/doc/fr/tutorial/programming.rst # 0 doctests failed
sage -t -long devel/sage/sage/schemes/plane_curves/constructor.py # 0 doctests
failed
sage -t -long devel/sage/sage/schemes/elliptic_curves/BSD.py # 1 doctests failed
sage -t -long devel/sage/sage/parallel/decorate.py # 0 doctests failed
sage -t -long devel/sage/sage/libs/galrep/wrapper.pyx # 0 doctests failed
----------------------------------------------------------------------
Total time for all tests: 3990.8 seconds

but the failure you observed, devel/sage/sage/misc/trace.py passed for me.

When I repeat the tests individually, they all pass except the last, which fails
with a SIGBUS error.


kirkby@t2:[/tmp/kirkby/sage-4.5.alpha4] $ ./sage -t -long

devel/sage/sage/libs/galrep/wrapper.pyx
sage -t -long "devel/sage/sage/libs/galrep/wrapper.pyx"


------------------------------------------------------------
Unhandled SIGBUS: A bus error occurred in Sage.
This probably occurred because a *compiled* component
of Sage has a bug in it (typically accessing invalid memory)
or is not properly wrapped with _sig_on, _sig_off.
You might want to run Sage under gdb with 'sage -gdb' to debug this.
Sage will now terminate (sorry).
------------------------------------------------------------


[14.6 s]

----------------------------------------------------------------------
The following tests failed:


sage -t -long "devel/sage/sage/libs/galrep/wrapper.pyx"
Total time for all tests: 14.6 seconds


I decided to change to your directory John, and run the tests from your build,
rather than mine.


kirkby@t2:[/tmp/kirkby/sage-4.5.alpha4] $ cd /home/palmieri/t2/sage-4.5.alpha4
kirkby@t2:[/home/palmieri/t2/sage-4.5.alpha4] $ ./sage -t -long
devel/sage/sage/libs/galrep/wrapper.pyx
sage -t -long "devel/sage/sage/libs/galrep/wrapper.pyx"


------------------------------------------------------------
Unhandled SIGBUS: A bus error occurred in Sage.
This probably occurred because a *compiled* component
of Sage has a bug in it (typically accessing invalid memory)
or is not properly wrapped with _sig_on, _sig_off.
You might want to run Sage under gdb with 'sage -gdb' to debug this.
Sage will now terminate (sorry).
------------------------------------------------------------


[70.0 s]

----------------------------------------------------------------------
The following tests failed:


sage -t -long "devel/sage/sage/libs/galrep/wrapper.pyx"
Total time for all tests: 70.1 seconds
kirkby@t2:[/home/palmieri/t2/sage-4.5.alpha4] $


Then I decided to clearly out my $HOME/.sage, and re-run the test. This time it
worked ok


So it looks like some junk left over one build/test of Sage can cause problems
when builing/testing a later one.

It's hard to know what to make of all this.


Dave

John H Palmieri

unread,
Jul 7, 2010, 11:07:10 AM7/7/10
to sage-release
On Jul 6, 9:22 pm, William Stein <wst...@gmail.com> wrote:
> On Wed, Jul 7, 2010 at 6:12 AM, William Stein <wst...@gmail.com> wrote:
> > On Wed, Jul 7, 2010 at 6:09 AM, Pat LeSmithe <qed...@gmail.com> wrote:
> >> On 07/06/2010 11:36 AM, William Stein wrote:
> >>> Has anybody successfully got all tests to pass for sage-4.5.alpha[3-4]
> >>> on sage.math.washington.edu?  For me, many, many maxima-related tests
> >>> fail.
>
> >> I have seen all tests pass with 4.5.alpha{1,3,4}, but I've also noticed
> >> many Maxima-related failures on occasion on sage.math with
> >> SAGE_PARALLEL_SPKG_BUILD="yes" and 4, 6, or 12 parallel jobs.  Two
> >> representative errors are mentioned at
>
> >>http://trac.sagemath.org/sage_trac/ticket/9274#comment:17
>
> > Yes, that's the problem I'm seeing.
>
> > Note that I am *not* building with SAGE_PARALLEL_SPKG_BUILD set...

I was all ready to blame SAGE_FAT_BINARY. But now I have now built
4.5.alpha4 four times on sage.math, all with MAKE='make -j8' and then
with the following different combinations:

SAGE_FAT_BINARY unset
SAGE_PARALLEL_SPKG_BUILD unset

SAGE_FAT_BINARY unset
SAGE_PARALLEL_SPKG_BUILD='yes'

SAGE_FAT_BINARY='yes'
SAGE_PARALLEL_SPKG_BUILD unset

SAGE_FAT_BINARY='yes'
SAGE_PARALLEL_SPKG_BUILD='yes'

Once or twice I had the failure with startup.py, but that's the only
doctest failure I've seen. No problems with maxima at all. So I have
no idea what's causing the problems William has seen. Can there be
something wonky in your .sage directory? Three of the builds still
exist: the last three, in, respectively

/scratch/palmieri/sage-4.5.alpha4/
/scratch/palmieri/tmp/sage-4.5.alpha4/
/scratch/palmieri/tmp-serial/sage-4.5.alpha4/

(Sorry about the nondescriptive names, but that's how I set it up, and
once I had started building, I didn't want to rename the directories
because I didn't want to trigger whatever Pat has seen when you move
the sage build around.)

--
John

John H Palmieri

unread,
Jul 7, 2010, 11:48:31 AM7/7/10
to sage-release
On Jul 7, 2:43 am, "Dr. David Kirkby" <david.kir...@onetel.net> wrote:
> On 07/ 7/10 02:59 AM, John H Palmieri wrote:
>
> > On t2.math, one failure:
>
> > sage -t  -long devel/sage/sage/misc/trace.py
>
> This is odd, as this test passes for me on 't2'.

Well, it failed the first three times I tested it, but it's now passed
several times in a row. So I'm downgrading the ticket from
"blocker". It's not clear what to do about it at all, since it is not
reproducible.

> One possible reason for your failure is that the configuration of the ZFS pools
> on 'disk.math' have been modified (not by me I would add) to increase NFS
> performance by disabling a logging device - a practice considered unsafe.

I would be happy to build on some other disk. Where do you recommend?
A few months ago, there was a shortage of disk space in /scratch, so
I've been reluctant to build there. Has that problem been cleared up?

--
John

Dr. David Kirkby

unread,
Jul 7, 2010, 2:01:17 PM7/7/10
to sage-r...@googlegroups.com
On 07/ 7/10 04:48 PM, John H Palmieri wrote:
> On Jul 7, 2:43 am, "Dr. David Kirkby"<david.kir...@onetel.net> wrote:
>> On 07/ 7/10 02:59 AM, John H Palmieri wrote:
>>
>>> On t2.math, one failure:
>>
>>> sage -t -long devel/sage/sage/misc/trace.py
>>
>> This is odd, as this test passes for me on 't2'.
>
> Well, it failed the first three times I tested it, but it's now passed
> several times in a row. So I'm downgrading the ticket from
> "blocker". It's not clear what to do about it at all, since it is not
> reproducible.

I may be wrong, but I get the feeling there are a number of tests that seem to
work for one person, but not another, on the same machine.

I for example can't get BSD.py to work on t2 for me.

I'd leave it open for now. There is certainly something odd about the doc tests
- see my post on sage-devel and the interesting followup from Simon.

>> One possible reason for your failure is that the configuration of the ZFS pools
>> on 'disk.math' have been modified (not by me I would add) to increase NFS
>> performance by disabling a logging device - a practice considered unsafe.
>
> I would be happy to build on some other disk. Where do you recommend?
> A few months ago, there was a shortage of disk space in /scratch, so
> I've been reluctant to build there. Has that problem been cleared up?

I've been a bit ruthless and cleared out /scratch quite a bit. A lot of people
had a 4.3.0.1 tar file decompressed in /scratch. Given its quite old, and the
tar file may be found in /usr/local, I've deleted them all.

There's now 15 GB free. But be aware that building packages in parallel means
several are decompressed at one time. I don't know what the maximum disk space
might be in a such a case, but it will clearly be more than for serial builds.

> --
> John
>

John H Palmieri

unread,
Jul 8, 2010, 10:52:20 AM7/8/10
to sage-release
On Jul 6, 5:30 am, Robert Miller <r...@rlmiller.org> wrote:
> http://sage.math.washington.edu/home/release/sage-4.5.alpha4/sage-4.5...

I tried building this on the skynet machines, and it worked fine on
most of them.

- mark (sparc-SunOS-ultrasparc3): failed to build gnutls. I've never
been able to build on this machine.

- fulvia (x86_64-SunOS-core2): failed to build (no gnu tar). I've
never been able to build on this one, either.

- eno (x86_64-Linux-core2-fc), taurus (x86_64-Linux-nehalem-fc):
failed to build gfan, both with SAGE64 unset and with SAGE64='yes' .
**This is an issue which needs fixing**.

make[2]: Leaving directory `/home/palmieri/taurus/sage-4.5.alpha4/
spkg/build/gfan-0.4plus.p1/src'
rm: cannot remove `gfan_*': No such file or directory
./gfan: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not
found (required by ./gfan)
gfan links not created correctly


all tests passed:

- cicero (x86-Linux-pentium4-fc)
- cleo (ia64-Linux-rhel)
- flavius (x86_64-Linus-k8)
- iras (ia64-Linux-suse)
- lena (x86_64-Linux-k10)
- menas (x86_64-Linux-core2-suse)
- sextus (x86_64-Linux-netburst-fc)

(Actually, a few of these had doctest failures, but they were not
repeatable. I had this set up where all of them were sharing the
same .sage directory, and I wouldn't be surprised if this caused some
clashes which led to failures.)

--
John

David Kirkby

unread,
Jul 8, 2010, 11:17:29 AM7/8/10
to sage-r...@googlegroups.com
On 8 July 2010 15:52, John H Palmieri <jhpalm...@gmail.com> wrote:
> On Jul 6, 5:30 am, Robert Miller <r...@rlmiller.org> wrote:
>> http://sage.math.washington.edu/home/release/sage-4.5.alpha4/sage-4.5...
>
> I tried building this on the skynet machines, and it worked fine on
> most of them.
>
> - mark (sparc-SunOS-ultrasparc3): failed to build gnutls.  I've never
> been able to build on this machine.

I'll take a look at that. I think the issue may be that gcc was
configured to GNU linker or assembler. Using the GNU linker is
especially bad news on SPARC.

> - fulvia (x86_64-SunOS-core2): failed to build (no gnu tar).  I've
> never been able to build on this one, either.

/usr/sfw/bin/gtar will probably be the location of a GNU version of
tar. /usr/sfw/bin/gmake will probably be a GNU make. One needs to mess
around a bit with the path on Solaris, and copy some of these tools to
another directory.

> all tests passed:
>
> - cicero (x86-Linux-pentium4-fc)
> - cleo (ia64-Linux-rhel)
> - flavius (x86_64-Linus-k8)
> - iras (ia64-Linux-suse)
> - lena (x86_64-Linux-k10)
> - menas (x86_64-Linux-core2-suse)
> - sextus (x86_64-Linux-netburst-fc)
>
> (Actually, a few of these had doctest failures, but they were not
> repeatable.  I had this set up where all of them were sharing the
> same .sage directory, and I wouldn't be surprised if this caused some
> clashes which led to failures.)
>
> --
> John

I'm puzzled why so many people get non-reproducible doctest results. I
know for a fact you, me and William have all had failures which are
not reproducible.

I believe there are problems in the doctesting code.

http://trac.sagemath.org/sage_trac/ticket/9449

shows some tests which are not obvious if they are listed as passes or
failures. In particular, when one sees a summary that includes things
like

The following tests failed:

sage -t -long devel/sage/doc/fr/tutorial/programming.rst # 0
doctests failed
sage -t -long
devel/sage/sage/schemes/plane_curves/constructor.py # 0 doctests
failed


where there are 0 doctest failures in both of these failed tests, then
something is very wrong. I'm not sure how much I can trust the results
from the doctests

Dave

leif

unread,
Jul 8, 2010, 12:51:38 PM7/8/10
to sage-r...@googlegroups.com
Robert Miller wrote:
> http://sage.math.washington.edu/home/release/sage-4.5.alpha4/sage-4.5.alpha4.tar
>
> This is a short set of additions to alpha3, and I was going to start
> merging patches to sage-main before releasing, but because of
>
> http://trac.sagemath.org/sage_trac/ticket/9435
>
> I wanted to release this separately, since there are changes to deps,
> etc. Sorry about the rapid-fire alphas...
>
> SPKG changes are now frozen. Only blocker spkg tickets will be merged
> from now until sage-4.5 final. The next release will be rc0.


Ubuntu 9.04 x86_64 (Core2 quad, gcc 4.3.3, default 64-bit builds*):

I've successfully built Sage 4.5.alpha4 with SageNB 0.8.1 (#9430) and
zodb3 3.7.0.p4 (#9436), both sequentially and in parallel
(SAGE_PARALLEL_SPKG_BUILD="yes", MAKE="make -j4").

No downloads during install, HTML documentation builds (except missing
conf.py in thematic_tutorials), and all long tests passed (make ptestlong).

All builds/tests performed independently, i.e. from scratch.


I've given #9436 (4.5 blocker) positive review; left #9430 (major
enhancement) as "needs review" s.t. hopefully others test on other
platforms, too.


-Leif


http://trac.sagemath.org/sage_trac/ticket/9430 ("Update SageNB to
version 0.8.1")
http://trac.sagemath.org/sage_trac/ticket/9436 ("zodb3 causes downloads
during installation")
__________
* CFLAGS=CXXFLAGS="-march=native -fno-strict-aliasing",
CPPFLAGS="-march=native"

John H Palmieri

unread,
Jul 8, 2010, 4:00:50 PM7/8/10
to sage-release
On Jul 8, 8:17 am, David Kirkby <david.kir...@onetel.net> wrote:
> On 8 July 2010 15:52, John H Palmieri <jhpalmier...@gmail.com> wrote:
>
> > On Jul 6, 5:30 am, Robert Miller <r...@rlmiller.org> wrote:
> >>http://sage.math.washington.edu/home/release/sage-4.5.alpha4/sage-4.5...
>
> > I tried building this on the skynet machines, and it worked fine on
> > most of them.
>
> > - mark (sparc-SunOS-ultrasparc3): failed to build gnutls.  I've never
> > been able to build on this machine.
>
> I'll take a look at that. I think the issue may be that gcc was
> configured to GNU linker or assembler. Using the GNU linker is
> especially bad news on SPARC.

Please keep me informed if you figure anything out.

> > - fulvia (x86_64-SunOS-core2): failed to build (no gnu tar).  I've
> > never been able to build on this one, either.
>
> /usr/sfw/bin/gtar will probably be the location of a GNU version of
> tar. /usr/sfw/bin/gmake will probably be a GNU make. One needs to mess
> around a bit with the path on Solaris, and copy some of these tools to
> another directory.

I tried playing with this briefly in the past, but not recently.
Maybe I'll try again soon.

> I'm puzzled why so many people get non-reproducible doctest results. I
> know for a fact you, me and William have all had failures which are
> not reproducible.

I have some guesses for the particular cases here: running doctests in
parallel may cause memory issues, so some tests may time out when they
wouldn't when run on their own. I also had things very badly
configured: my account on the skynet machines all share the same home
directory, and I was building and doctesting sage on seven or eight
different machines at once. If they happen to try to write to the
same file in the .sage directory, there could be problems. I just
changed things so that when I log in, I set DOT_SAGE to ".sage_
$HOSTNAME" or something like that, in hopes of avoiding such problems
in the future.

> I believe there are problems in the doctesting code.

I wouldn't be surprised.

> http://trac.sagemath.org/sage_trac/ticket/9449
>
> shows some tests which are not obvious if they are listed as passes or
> failures. In particular, when one sees a summary that includes things
> like
>
> The following tests failed:
>
>     sage -t     -long devel/sage/doc/fr/tutorial/programming.rst # 0
> doctests failed
>     sage -t     -long
> devel/sage/sage/schemes/plane_curves/constructor.py # 0 doctests
> failed
>
> where there are 0 doctest failures in both of these failed tests, then
> something is very wrong. I'm not sure how much I can trust the results
> from the doctests

Well, it's much better to have files reported as failing when they
don't, rather than the other way around.

--
John

leif

unread,
Jul 8, 2010, 4:53:23 PM7/8/10
to sage-r...@googlegroups.com
John H Palmieri wrote:
> Well, it's much better to have files reported as failing when they
> don't, rather than the other way around.

I was thinking the same... ;-)

But if the former happens, can we be sure the latter does not?


-Leif

Dr. David Kirkby

unread,
Jul 8, 2010, 7:12:01 PM7/8/10
to sage-r...@googlegroups.com
On 07/ 8/10 09:00 PM, John H Palmieri wrote:
> On Jul 8, 8:17 am, David Kirkby<david.kir...@onetel.net> wrote:
>> On 8 July 2010 15:52, John H Palmieri<jhpalmier...@gmail.com> wrote:
>>
>>> On Jul 6, 5:30 am, Robert Miller<r...@rlmiller.org> wrote:
>>>> http://sage.math.washington.edu/home/release/sage-4.5.alpha4/sage-4.5...
>>
>>> I tried building this on the skynet machines, and it worked fine on
>>> most of them.
>>
>>> - mark (sparc-SunOS-ultrasparc3): failed to build gnutls. I've never
>>> been able to build on this machine.
>>
>> I'll take a look at that. I think the issue may be that gcc was
>> configured to GNU linker or assembler. Using the GNU linker is
>> especially bad news on SPARC.
>
> Please keep me informed if you figure anything out.

I did ask Mariah to build a gcc with the Sun linker and assembler. She has done
so and sent me this email a few weeks ago. I forgot about this, otherwise I
would have said earlier today.

======== From Mariah Lenox, system admin, skynet ======
I have built in /usr/local/gcc-4.4.3/sparc-SunOS-ultrasparc3-usrccsbin
gcc-4.4.3 using --with-build-time-tools=/usr/ccs/bin as David
suggested. You will have to put it in your path (and edit
your LD_LIBRARY_PATH) to pick it up.

Let me know if you need anything else.
-- Mariah
=========================================================

I recently found out its possible to put gcc in an arbitrary directory on
Solaris, without setting LD_LIBRARY_PATH, which makes things a lot less hassle
in my opinion. I've built gcc on one of my own SPARC like that, and it works well.

http://jblopen.com/node/16

I've not used skynet much - basically since the processors there are only 1.28
GHz and its little faster than my own 1.20 GHz processors. That said, since I
got the Sun Ultra 27 (3.33 GHz Xeon), I've not had the SPARCs switched on as
much. My slower (900 MHz one) is in the garage which I use as a heater in
winter. The faster (1200 MHz one) is in my house, but having that on is the last
thing I want now in the UK, with the outside temperature around 29 deg C.

Unlike the US, air conditioning is quite rare in domestic properties in the UK.
The weather barely ever warrants it, but it does just now. So having my 1200 MHz
SPARC running is defiantly a no-no just now.

It would be great if William could get a decent machine for building Sage,
though since building packages in parallel is working, things are a lot more
usable on 't2'.

>>> - fulvia (x86_64-SunOS-core2): failed to build (no gnu tar). I've
>>> never been able to build on this one, either.
>>
>> /usr/sfw/bin/gtar will probably be the location of a GNU version of
>> tar. /usr/sfw/bin/gmake will probably be a GNU make. One needs to mess
>> around a bit with the path on Solaris, and copy some of these tools to
>> another directory.
>
> I tried playing with this briefly in the past, but not recently.
> Maybe I'll try again soon.

It should work. The main obstacles to overcome are

* The first gcc in your path needs to be a recent one.
* The first 'make' in your path needs to be the GNU one. That means you need to
copy the /usr/sfw/bin/gmake to 'make'.
* The first 'tar' in your path need to be the GNU one. Again, you need to copy
the /usr/sfw/bin/gtar to 'tar'
* The first 'ld' in your path needs to be the Sun one (/usr/ccs/bin/ld)

You really need to create a 'bin' directory, put some things there, and have
your paths set correctly. You could just look at the paths on t2, and see what I
set up there.

To make a binary distribution, you need to have the GNU version of 'cp' in your
path first. That would need to be built from source I expect, unless 'coreutils'
is installed (at least I think that's in coreutils).

>> I'm puzzled why so many people get non-reproducible doctest results. I
>> know for a fact you, me and William have all had failures which are
>> not reproducible.
>
> I have some guesses for the particular cases here: running doctests in
> parallel may cause memory issues,

True.

> so some tests may time out when they
> wouldn't when run on their own.

Possibly, if they swapped. Though the problems I've seen are not timeouts. I've
got SAGE_TIMEOUT set to 1000 s and SAGE_TIMEOUT_LONG set to 10000 s.

> I also had things very badly
> configured: my account on the skynet machines all share the same home
> directory, and I was building and doctesting sage on seven or eight
> different machines at once.

This could still be the issue even if you use one machine, if multiple tests are
writing to the one file at the same time. That could certainly be the reason for
some of the problems I've seen. I know deleting $HOME/.sage permitted at least
one test to pass, which repeatidly failed before I deleted that directory.


> If they happen to try to write to the
> same file in the .sage directory, there could be problems.

Yes, as you say, that could be. I think it is quite probable this is happening.

> I just
> changed things so that when I log in, I set DOT_SAGE to ".sage_
> $HOSTNAME" or something like that, in hopes of avoiding such problems
> in the future.

But if multiple tests on the same machine write to the .sage_$HOSTNAME, that
could still be an issue. Unfortunately 'make testlong' takes an eternity on
't2', so doing the tests in parallel is almost essential.

>> I believe there are problems in the doctesting code.
>
> I wouldn't be surprised.

Luckily some people seem to be working on this. There are a lot of doctest
related patches around. Let's hope some of them improve things.

>> http://trac.sagemath.org/sage_trac/ticket/9449
>>
>> shows some tests which are not obvious if they are listed as passes or
>> failures. In particular, when one sees a summary that includes things
>> like
>>
>> The following tests failed:
>>
>> sage -t -long devel/sage/doc/fr/tutorial/programming.rst # 0
>> doctests failed
>> sage -t -long
>> devel/sage/sage/schemes/plane_curves/constructor.py # 0 doctests
>> failed
>>
>> where there are 0 doctest failures in both of these failed tests, then
>> something is very wrong. I'm not sure how much I can trust the results
>> from the doctests
>
> Well, it's much better to have files reported as failing when they
> don't, rather than the other way around.
>
> --
> John

But if tests pass and are reported as failures, it gives me little confidence
that tests might fail and be reported as passes. I think someone has identified
some really horrible programming in some doctesting code. (It's mentioned on one
of the tickets about doctests). Hopefully that can be fixed. From viewing some
of the comments on tickets, there is a lot of code in the doctesting which is
badly written. At least its been identified, and will hopefully be fixed.

I know the following is not going to be a popular view, but I'm going to make
the point anyway.

Sage has done a lot to lower the barrier of computing ability needed to
contribute to Sage. Many see this as good - it allows more contributors. I'm not
convinced that is such a good idea myself. This is my opinion is resulting in
some dubious code. People with poor programming skills are reviewing the code of
those with poor programming skills.

There is one Sage developer, (who shall remain nameless), who put in an email
sent to half a dozen people (myself included) that he did not want to waste time
thinking about how code might fail - he would rather apply patches when there
are bug reports. Anyone with a computer background would know this is poor
practice.

Of course everyone makes mistakes - I know I have. The review process is not
perfect and never will be, just like peer review of academic papers is not
perfect and some really poor work does get published. But at least the academic
peer review process normally results in the paper only being sent to someone
with a record of publications in that field - i.e. someone competent in that
area. That's not the case with Sage, where poor programmers can review the code
of poor programmers.

I think any belief that being a good mathematician means you can contribute code
to Sage is seriously flawed. Yet that does appear to be a common view.

Dave

Nathann Cohen

unread,
Jul 8, 2010, 10:03:13 PM7/8/10
to sage-r...@googlegroups.com
> There is one Sage developer, (who shall remain nameless), who put in an
> email sent to half a dozen people (myself included) that he did not want to
> waste time thinking about how code might fail - he would rather apply
> patches when there are bug reports. Anyone with a computer background would
> know this is poor practice.

I don't like to remain nameless.

Very often I look at my code and wonder how it could be used, and
misused -- what you may call debugging it, and I like that -- but I
have to admit I can not get myself to care about different
architectures... I was thinking about platform-related failures when I
wrote that, though it did not appear.

I never "learned" programing. I just found myself in high school,
spending nights in front of a computer, that's all. I first learned to
use windows, deal with its incoherences, then later linux, but I never
trusted computers. I trust them less and less, the more I get to know
them. And most importantly, when I use Sage I don't expect it to be
always right (and not only because of the functions I wrote myself, if
this is what you thought about :-D). Sometimes I use its methods in
long code, and my code doesn't work. I debug, I debug, I debug, then I
find out the poblem does not come from my own code, but from Sage's
code itself.

I blame no one, that's how computers are. That's how software is.
Usually, I re-read the code, and I understand what the initial
programmer was thinking about, and what -- and why -- he did not see
at that time. I write a patch, call it settled, and use Sage again.

"Try and fail" is just how everything around you has been built,
studied, and thought. That does not mean you are not to pay attention
to what you do, it just means that you know somewhere that even if you
were very smart and had a very long experience, you can never think of
everything.

And also that most of the time, you learn much quicker by trying and
failing, than by not trying until you are sure to be right... So as
long as you are doing your job honestly, not stopping debugging
because of lazyness but because you feel you tied everything you could
think of, I call it good work. If it fails at some point, it was still
good work, and now your work is to patch it.

All in all, nothing that I jut said is very interesting nor original.
I mainly did not want to stay silent when I am mentionned, even
anonymously. I also wanted to add something at this subject. I know I
am sometimes exasperating, and it is not totally unconsciously, as I
often hope people with differents view will be glad to come and
discuss them with me -- it always teaches something new. Sometimes, I
even change my original views (yes, yes !). Some other times, people
do not want to talk, and get quickly angry. It happens, I do not mind.

I would like to ask a favor from you, though. I try to live without
having to hide what I do, nor what I say. It means taking many
decisions based on morals (at least my notion of it), which is very
often a self-destructive path, for instance in the word of research.
When I am wrong, and if I happen to notice it, I try to make it plain
to all the people that were concerned. That's why the part I liked the
least in your message was the very fact that my name did not appear.
Please do not do that anymore.

I am a loud guy, I fear above all to be muted :-)

Oh, and another thing. One I already mentionned once, but I do not
remember when :

Sage is a Free Software. The good side of it is that everybody can
help. The bad side of it is, sadly, that really *everybody* can help.
What I really like in the Sage community is that whenever I made
mistakes in the code I sent, I received both a possible fix and an
explanation of why it was a mistake, sometimes links toward
explanations, well -- teachings. I improved a lot.

I don't, and nobody does, when you answer "ask google" or complain on
mailing lists. Please, work with us -- not against us.

Nathann

Dr. David Kirkby

unread,
Jul 9, 2010, 12:38:02 AM7/9/10
to sage-r...@googlegroups.com
On 07/ 9/10 03:03 AM, Nathann Cohen wrote:

> I never "learned" programing. I just found myself in high school,
> spending nights in front of a computer, that's all. I first learned to
> use windows, deal with its incoherences, then later linux, but I never
> trusted computers.

I'm actually rather surprised you did not learn programming. It was part of my
electrical engineering degree which I got in 1986. Computers were a lot less
common back in the mid 1980s than they are today.

But the main point is, that you admit you did not learn programming, yet are in
the position to review the codes of others who quite possibly have little
ability at programming. IMHO, that situation is not ideal. That is not a
reflection on you as a person or your programming knowledge, but of the Sage
development process.

Of course, even if you did not learn programming formally, that does not mean
you can't read books about best practices and learn on your own. I am not
suggesting that Sage developers should have computer science degrees - though I
feel a few people with a professional background in software development would
be useful.

Perhaps the Sage review process should identify those with good software
background, and ensure that in circumstances where the programmer is
inexperienced, at least one reviewer who is competent. In other words, perhaps
in such cases there should be two reviewers - one with the mathematical
knowledge, another with the computer knowledge. That does happen in some cases,
but there are no formal means to ensure it happens - it is a bit of a hit and
miss affair.

Experience does not mean competence, though one tends to get more competent with
more experience.

Of course, finding two reviewers will be harder than finding one. There is
already a problem with a lack of code being reviewed - hence William writing
"The Sage Nagbot" (see sage-devel). Ultimately that might result in less code
committed to Sage, as developers would have to spend more time reviewing, and
less time writing code.

Is that a bad thing? I don't think it would be myself, if it resulted in higher
quality code. Others might disagree.

I'm not suggesting it's practical to devote the resources NASA would do to
software on a manned space flight, but I feel there needs to be more emphasis on
quality.

Your own failure to look at error codes was spotted in the review process. Other
fairly obvious failures to write good code are not, and do appear in Sage.

> I trust them less and less, the more I get to know
> them. And most importantly, when I use Sage I don't expect it to be
> always right (and not only because of the functions I wrote myself, if
> this is what you thought about :-D). Sometimes I use its methods in
> long code, and my code doesn't work. I debug, I debug, I debug, then I
> find out the poblem does not come from my own code, but from Sage's
> code itself.

Sage will never be bug free. As you have learned, you can never 100% trust
software. There's a rather famous book on software development by Brooks, called
The Mythical Man-Month

http://en.wikipedia.org/wiki/The_Mythical_Man-Month

One of Brook's ideas presented in the book is that there's a tendency to an
irreducible number of errors.

http://en.wikipedia.org/wiki/The_Mythical_Man-Month#The_tendency_towards_irreducible_number_of_errors

For any sufficiently complex piece of software (and Sage would be considered
complex), any attempt to fix observed errors tends to result in the introduction
of other errors.

But there are techniques to make software less buggy - like checking error
codes, rather than just assuming things had worked, which was the particular
thing I took issue with. The Sage Developers guide clearly shows how you should
check the error code of 'configure' before running 'make' and check the error
code of 'make' before running 'make install'.


> I blame no one, that's how computers are. That's how software is.

Non-trivial software will always have bugs. But different development techniques
can have a dramatic effect on the frequency of those bugs.

> Usually, I re-read the code, and I understand what the initial
> programmer was thinking about, and what -- and why -- he did not see
> at that time. I write a patch, call it settled, and use Sage again.

That I'm afraid is far from an optimal technique.

> "Try and fail" is just how everything around you has been built,
> studied, and thought.

There is also the possibility of designing well in the first place, which will
reduce this trial and error process somewhat. I'll give you a quote which is at
least partially true

"The sooner you start to code, the longer the program will take"

http://www.linfo.org/q_programming.html

Of course, in the limit that is most obviously wrong, as if you delayed coding
forever, the program would not be finished instantly. But it is generally
helpful to think about the problem, before writing the code.

> That does not mean you are not to pay attention
> to what you do, it just means that you know somewhere that even if you
> were very smart and had a very long experience, you can never think of
> everything.

True. I'm not however talking about people being smart, but rather have a
reasonable understanding of best practices and what they are doing.

> And also that most of the time, you learn much quicker by trying and
> failing, than by not trying until you are sure to be right...

That's no doubt true, but it's debatable whether the code should be committed to
Sage when the programmer has minimal software development skills - at least not
without it being thoroughly reviewed by someone who is competent. That person is
then likely to suggest many changes.

> So as
> long as you are doing your job honestly, not stopping debugging
> because of lazyness but because you feel you tied everything you could
> think of, I call it good work. If it fails at some point, it was still
> good work, and now your work is to patch it.

Honestly, you (and the Sage project), would gain if you took some time to read
about best practices in software development. You are not the only one in this
situation.

Perhaps there should a section in the Sage Developers Manual where some
information on this subject can be found, as it is clearly lacking in some Sage
developers. Unfortunately, the best resources on this I know of are not free.

> All in all, nothing that I jut said is very interesting nor original.
> I mainly did not want to stay silent when I am mentionned, even
> anonymously.

That's fine, but I did give you the opportunity to remain anonymous. My points
were not aimed at you in particular. That said, I think you revealing your
identity, and me replying, will hopeful be good for Sage, as issues of how Sage
is developed need to be discussed.

> I also wanted to add something at this subject. I know I
> am sometimes exasperating, and it is not totally unconsciously, as I
> often hope people with differents view will be glad to come and
> discuss them with me -- it always teaches something new. Sometimes, I
> even change my original views (yes, yes !). Some other times, people
> do not want to talk, and get quickly angry. It happens, I do not mind.

I guess I let of steam over this, when it appeared that you really did not want
to think about how your code could fail, but would rather have applied patches
when there were bug reports. This really is not good practice. If you can't see
that, then you should take some time to read about best practices.

> I would like to ask a favor from you, though. I try to live without
> having to hide what I do, nor what I say. It means taking many
> decisions based on morals (at least my notion of it), which is very
> often a self-destructive path, for instance in the word of research.
> When I am wrong, and if I happen to notice it, I try to make it plain
> to all the people that were concerned. That's why the part I liked the
> least in your message was the very fact that my name did not appear.
> Please do not do that anymore.

I did not mention your name. I don't think that was unreasonable. I felt you
deserved the opportunity to remain anonymous if you wanted to.

I was talking more in general, rather than you in particular. With a few rare
exceptions, what I'm writing in this email are not aimed at you in particular,
but the Sage development process in general.

> I am a loud guy, I fear above all to be muted :-)

I did not attempt to mute you. I just gave you the opportunity to remain anonymous.

> Oh, and another thing. One I already mentionned once, but I do not
> remember when :
>
> Sage is a Free Software.

Yes.

> The good side of it is that everybody can
> help.

Yes

> The bad side of it is, sadly, that really *everybody* can help.

If it means that people who are not competent programmers are writing code,
which gets reviewed by people who are not competent programmers, then that is
indeed bad. But it does not need to be like that.

> What I really like in the Sage community is that whenever I made
> mistakes in the code I sent, I received both a possible fix and an
> explanation of why it was a mistake, sometimes links toward
> explanations, well -- teachings. I improved a lot.

That's good, though some people (including myself), will tend to lose patience
if they feel you could find the information yourself easily.

> I don't, and nobody does, when you answer "ask google" or complain on
> mailing lists. Please, work with us -- not against us.

My point about suggesting you look at Google on a ticket was that quite honestly
you could have found the answer quickly with a bit of research yourself. I'm not
keen to waste my time to explain something you could find out for yourself
quickly. You will find a lot of people like me. Some will just say RTFM, or give
you a link like

http://www.justfuckinggoogleit.com/

I guess you see thinking about how software will fail to be a waste of your
time. I see answering questions that Google could answer a waste of my time.
Nobody wants to waste their time.

> Nathann

Dave

Pat LeSmithe

unread,
Jul 9, 2010, 12:44:11 AM7/9/10
to sage-r...@googlegroups.com
On 07/07/2010 10:07 AM, John H Palmieri wrote:
> On Jul 6, 9:22 pm, William Stein <wst...@gmail.com> wrote:
>> On Wed, Jul 7, 2010 at 6:12 AM, William Stein <wst...@gmail.com> wrote:
>>> On Wed, Jul 7, 2010 at 6:09 AM, Pat LeSmithe <qed...@gmail.com> wrote:
>>>> On 07/06/2010 11:36 AM, William Stein wrote:
>>>>> Has anybody successfully got all tests to pass for sage-4.5.alpha[3-4]
>>>>> on sage.math.washington.edu? For me, many, many maxima-related tests
>>>>> fail.
>>
>>>> I have seen all tests pass with 4.5.alpha{1,3,4}, but I've also noticed
>>>> many Maxima-related failures on occasion on sage.math with
>>>> SAGE_PARALLEL_SPKG_BUILD="yes" and 4, 6, or 12 parallel jobs. Two
>>>> representative errors are mentioned at
>>
>>>> http://trac.sagemath.org/sage_trac/ticket/9274#comment:17
>>
>>> Yes, that's the problem I'm seeing.
>>
>>> Note that I am *not* building with SAGE_PARALLEL_SPKG_BUILD set...
>
> I was all ready to blame SAGE_FAT_BINARY. But now I have now built
> 4.5.alpha4 four times on sage.math, all with MAKE='make -j8' and then
> with the following different combinations:

I've opened

http://trac.sagemath.org/sage_trac/ticket/9460

to track this.

Dr. David Kirkby

unread,
Jul 9, 2010, 1:08:46 AM7/9/10
to sage-r...@googlegroups.com


One possibility for this is that the code added to Sage to build packages in
parallel is changing the order in which packages are built.

Maxima only depends on the base packages and ECL directly, but ECL depends on
many other things. If, for example a system has the ntl library, and it is built
is Sage, then changing the order of packages could mean that Sage swaps from
using the system one, to one in Sage.

It would be worth trying to run the maxima tests many times on Solaris and OS X,
just to confirm if is indeed a Linux specific issue.

when I build Sage in parallel on OpenSolaris, I sometimes look at the system
monitor. I've never seen a lot of memory being used. My machine has 12 GB RAM,
but I don't think I've seen 2 GB used when building Sage and that's with the
windowing system running, firefox, thunderbird and a few terminals. So I don't
think building in parallel is using much more RAM, though clearly it will use
some more than building packages serially.


Dave

Robert Miller

unread,
Jul 9, 2010, 3:30:16 AM7/9/10
to sage-r...@googlegroups.com
David,

On Fri, Jul 9, 2010 at 4:03 AM, Nathann Cohen <nathan...@gmail.com> wrote:
> I don't [like it], and nobody does, when you answer "ask google" or complain on


> mailing lists. Please, work with us -- not against us.

I very much agree with this sentiment. General complaining about how
things aren't perfect is counterproductive. The "Sage way" is to
simply inform and improve in specific instances. Being a volunteer
project, we can't make demands on our developers or they may very well
leave. What you can do, however, is provide your expertise in specific
situations. E.g. just because a ticket has a positive review does not
necessarily mean it is ready for inclusion-- there are probably
several tickets listed with positive reviews which probably do need
work, and it is neither a waste of time nor stepping on anyone's toes
to provide feedback on these, or even set them to needs_work.

Message has been deleted

David Kirkby

unread,
Jul 9, 2010, 6:25:38 AM7/9/10
to sage-r...@googlegroups.com
On 9 July 2010 06:44, Nathann Cohen <nathan...@gmail.com> wrote:
> Hello !!! :-)

Hello

> Well, I just slept three hours and I read your answer immediately
> after waking up... I braced myself for something much more offensive
> and was quite surprised,

I don't wish to be offensive.

> actually you got me *very* curious by
> mentionning so many times what you call "best practices".... So now I
> only have one thing to do : you are talking about studying them -->
> can you point me to any such book/website ?

A couple of books I can think of are

http://www.amazon.com/Software-Engineering-9th-Ian-Sommerville/dp/0137035152/ref=dp_ob_title_bk

I've got an older copy of

http://www.amazon.com/Software-Engineering-Practitioners-Roger-Pressman/dp/0073375977/ref=sr_1_2?ie=UTF8&s=books&qid=1278667706&sr=1-2-spell

which I found very useful. The concensus seems to be that recent
copies have not kept up with newer techniques in software development.

Niether book is cheap, though university libraries should have these.
But I somewhat doubt you will want to read a several hundred page book
- I would not blame you!

Wikipedia has a good section on software testing

http://en.wikipedia.org/wiki/Software_testing

I would imagine reading the whole of that would take about an hour -
which would be an hour well spent.

This section seemed particulary relevent to one of the issues we discussed.

http://en.wikipedia.org/wiki/Software_testing#Finding_faults_early

Other links on related items.

http://www.joelonsoftware.com/articles/fog0000000043.html
http://on-agile.blogspot.com/2006/12/hidden-cost-of-delaying-bug-fixes.html

This following links is possiblly an intersting link to add to the
Sage Devlopers Guide about the review process. I've not read the page
myself, but it seem to cover the sort of things we should considered
when reviewing code.

http://www.processimpact.com/articles/revu_sins.html

Desite this being a page on a commerical company offering consulting
in software development best practices

http://www.construx.com/Page.aspx?nid=14

it does look like what are some useful link.


> I'd be quite glad to give it a look -!

There's a few ideas. Perhaps if you could look at a few, or Google terms like

software engineering
cost to fix bugs
softwave development best practices

and find a few, then see what you find most useful. I'm sure other
developers would like to knw of useful resources.

I personally don't feel qualified to write anythig on this topic to
add to the Devlopers Guide. However, if we could find some useful
links to web resources, that would be helpful.

> Thankssssssssssss ! :-)
>
> Nathann


Dave

Reply all
Reply to author
Forward
0 new messages