Occasional doctest failure in libs/fplll/fplll.pyx with 4.6.rc0

0 views
Skip to first unread message

Mitesh Patel

unread,
Oct 30, 2010, 6:14:17 PM10/30/10
to sage-...@googlegroups.com
I ran

./sage -t -long -force_lib "devel/sage/sage/libs/fplll/fplll.pyx"

1000 times in serial [1] with a 64-bit 4.6.rc0 built on OS X 10.6
(bsd.math). All but one of the runs pass. The failure:

Run 766 of 1000
Detected SAGE64 flag
Building Sage on OS X in 64-bit mode
sage -t -long -force_lib "devel/sage/sage/libs/fplll/fplll.pyx"
**********************************************************************
File
"/Users/buildbot/build/sage/bsd-2/bsd_64_full/build/sage-4.6.0pre0/devel/sa
ge/sage/libs/fplll/fplll.pyx", line 853:
sage: L.echelon_form() == A.echelon_form()
Expected:
True
Got:
False

Questions:

* Is this known?
* Why might the error occur on occasion?
* Does anyone see this on other platforms?

[1] Here's the script I used

#!/bin/bash

RUNS=1000
#for I in `seq $RUNS` # use this for non-Macs
for I in `jot $RUNS` # use for Macs
do
echo Run $I of $RUNS
./sage -t -long -force_lib devel/sage/sage/libs/fplll/fplll.pyx
done

David Kirkby

unread,
Oct 30, 2010, 6:35:46 PM10/30/10
to sage-...@googlegroups.com
On 30 October 2010 23:14, Mitesh Patel <qed...@gmail.com> wrote:
> I ran
>
> ./sage -t -long -force_lib "devel/sage/sage/libs/fplll/fplll.pyx"
>
> 1000 times in serial [1] with a 64-bit 4.6.rc0 built on OS X 10.6
> (bsd.math).  All but one of the runs pass.  The failure:
>
> Run 766 of 1000
> Detected SAGE64 flag
> Building Sage on OS X in 64-bit mode
> sage -t -long -force_lib "devel/sage/sage/libs/fplll/fplll.pyx"
> **********************************************************************
> File
> "/Users/buildbot/build/sage/bsd-2/bsd_64_full/build/sage-4.6.0pre0/devel/sa
> ge/sage/libs/fplll/fplll.pyx", line 853:
>    sage: L.echelon_form() == A.echelon_form()
> Expected:
>    True
> Got:
>    False
>
> Questions:
>
>  * Is this known?
>  * Why might the error occur on occasion?
>  * Does anyone see this on other platforms?

When I doctested Sage 100 times some months back there were several
failures, but this was not one of them.

I've just started a script on 'hawk' So far its been run 332 times
with no failures. It is taking 2.2s/test, so I'll let you know if
there is a failure, or if I get to 10,000 without a failure.

Dave

Mitesh Patel

unread,
Oct 30, 2010, 6:55:04 PM10/30/10
to sage-...@googlegroups.com

I get 6 failures out of 1000 runs with 4.6.rc0 on sage.math (64-bit
Ubuntu 8.04.4 LTS).

David Kirkby

unread,
Oct 30, 2010, 7:09:14 PM10/30/10
to sage-...@googlegroups.com

So far all passes of 1202 runs on OpenSolaris.

This is with very latest alpha I managed to find (sage-4.6.1.alpha0).
I'll try it with 4.6.rc0 too.

Dave

Mitesh Patel

unread,
Oct 30, 2010, 9:00:24 PM10/30/10
to sage-...@googlegroups.com

I get 5 failures out of 1000 runs with a 32-bit 4.6.rc0 on OS X 10.6
(bsd.math).

David Kirkby

unread,
Oct 31, 2010, 5:52:36 AM10/31/10
to sage-...@googlegroups.com
On 31 October 2010 00:09, David Kirkby <david....@onetel.net> wrote:

> So far all passes of 1202 runs on OpenSolaris.
>
> This is with very latest alpha I managed to find (sage-4.6.1.alpha0).
> I'll try it with 4.6.rc0 too.
>
> Dave
>

I got a number of failures, though not in the way you do.

On sage-4.6.rc0 I get

38 failures, 14837 passes, from 14875 tests.

But not a single one failed due to returning the wrong result, but
instead like this:

Run 442 of 100000


sage -t -long -force_lib "devel/sage/sage/libs/fplll/fplll.pyx"

python: can't open file '/export/home/drkirkby/.sage//tmp/fplll.py':
[Errno 2] No such file or directory

[0.2 s]

----------------------------------------------------------------------
The following tests failed:


sage -t -long -force_lib "devel/sage/sage/libs/fplll/fplll.pyx"

Total time for all tests: 0.2 seconds

Note this is on a local file system (128 bit ZFS). The discs are
mirrored and there have been zero detected errors in the disks. The
RAM is error correcting RAM used in a Sun workstation. The RAM is not
overclocked - in fact, it is run at 1067 MHz not the 1333 MHz it is
rated at, as the clock rate needs to be reduced if there is more than
6 GB in this machine (it has 12 GB RAM).

I've seen the "No such file or directory" issues before, and always
assumed it was due to parallel doctesting, but now I am not so sure.

Dave

leif

unread,
Oct 31, 2010, 6:10:02 AM10/31/10
to sage-devel
On 31 Okt., 02:00, Mitesh Patel <qed...@gmail.com> wrote:
> On 10/30/2010 06:09 PM, David Kirkby wrote:
> > On 30 October 2010 23:55, Mitesh Patel <qed...@gmail.com> wrote:
> >>>> "/Users/buildbot/build/sage/bsd-2/bsd_64_full/build/sage-4.6.0pre0/devel/sa
> >>>> ge/sage/libs/fplll/fplll.pyx", line 853:
> >>>>    sage: L.echelon_form() == A.echelon_form()
> >>>> Expected:
> >>>>    True
> >>>> Got:
> >>>>    False
> >> I get 6 failures out of 1000 runs with 4.6.rc0 on sage.math (64-bit
> >> Ubuntu 8.04.4 LTS).
>
> > So far all passes of 1202 runs on OpenSolaris.
>
> > This is with very latest alpha I managed to find (sage-4.6.1.alpha0).
> > I'll try it with 4.6.rc0 too.
>
> I get 5 failures out of 1000 runs with a 32-bit 4.6.rc0 on OS X 10.6
> (bsd.math).

3/1000 failures on Ubuntu 10.04 x86_64 (Core2), Sage 4.6.rc0, first
run.
4/1000 failures on Ubuntu 10.04 x86_64 (Core2), Sage 4.6.rc0, second
run.
3/1000 failures on Ubuntu 9.04 x86_64 (Core2), Sage 4.6.rc0, first
run.
2/ 500 failures on Ubuntu 9.04 x86_64 (Core2), Sage 4.6.rc0, second
run.
5/1000 failures on Ubuntu 9.04 x86 (Pentium 4 Prescott), Sage 4.6.rc0.

(Exactly the same as above, line 853, False instead of True.)

P.S.:

#!/usr/bin/env bash

case $# in
1) runs=$1;;
0) runs=1000;;
*) echo "Usage: $0 [num_runs]"
exit 1
esac

num_failed=0
failed_runs=""

for ((r=1;$r<=$runs;r=$r+1))
do
echo "Run $r/$runs:"
./sage -t -long -force_lib devel/sage/sage/libs/fplll/fplll.pyx
if [ $? -ne 0 ]; then
num_failed=`expr $num_failed + 1`
failed_runs="$failed_runs $r"
fi
done
echo "Failures: $num_failed/$runs"
if [ $num_failed -ne 0 ]; then
echo "Failed runs:$failed_runs"
fi

(One could of course make the file to test an optional parameter,
too, ...) :-)


-Leif

David Kirkby

unread,
Oct 31, 2010, 6:10:26 AM10/31/10
to sage-...@googlegroups.com
On 31 October 2010 09:52, David Kirkby <david....@onetel.net> wrote:

> python: can't open file '/export/home/drkirkby/.sage//tmp/fplll.py':

I just realised why. The doctest is writing to $HOME/.sage. But I was
doctests two versions of Sage (one based on 4.6.rc0, and another based
on sage-4.6.1.alpha0. These were both writing to the same directory,
so in effect it was a result of me testing in parallel.

So whilst I get failures, these can be explained by using the same
directory for temporary files.

So actually I have:

15323 correct results using sage-4.6.rc0
16425 correct results using sage-4.6.1.alpha0
0 incorrect results
69 failures in sage-4.6.1.alpha0 due to "No such file or directory" issues
40 failures in /sage-4.6.rc0 due to "No such file or directory" issues

So basically, there are 31748 and not a single incorrect result. The
only failures are likely to be a result of using the same name for a
temporary file, which happened because I was doctesting in two
directories at the same time.

Dave

Mitesh Patel

unread,
Oct 31, 2010, 6:12:28 AM10/31/10
to sage-...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages