Checking all but Python self-tests

[kcrisman]

Here is a probably dumb question.

Currently (as far as I am aware), most spkgs should pass with
SAGE_CHECK set, though it does depend on the platform and specific
machine. Python is somewhat notorious for not doing so. I'd like to
be able to start using SAGE_CHECK while building, but currently this
is annoying because it means I have to wait until it fails with
Python, then unset it for Python, reset it on the next spkg... not
very automated.

Could there be yet another variable SAGE_CHECK_PYTHON or something
which is for checking just that, or SAGE_CHECK_NOT_PYTHON which
doesn't, or...? I realize there are all kinds of annoying issues
here.

Maybe a minimal way would be to introduce SAGE_CHECK_PYTHON, and then
if one has SAGE_CHECK set during the Python build it gives a message
about how to "really" check its self-tests, with no other behavior
changed.

Alternately, if there actually is an easy way to use SAGE_CHECK for
everything except Python, please let me know.

Thanks!
- kcrisman

[John H Palmieri]

On Tuesday, January 17, 2012 10:06:18 AM UTC-8, kcrisman wrote:

Here is a probably dumb question.

Currently (as far as I am aware), most spkgs should pass with
SAGE_CHECK set, though it does depend on the platform and specific
machine.  Python is somewhat notorious for not doing so.  I'd like to
be able to start using SAGE_CHECK while building, but currently this
is annoying because it means I have to wait until it fails with
Python, then unset it for Python, reset it on the next spkg... not
very automated.

Slightly more automated, but still some work, is to create a new Python spkg file: just deleting the spkg-check file should be good enough.  So unpack python-....spkg, delete spkg-check, run sage --pkg python-... (you don't even need to commit the changes), and then do 'make'.

--
John

[Volker Braun]

I consider it a bug that Python's testsuite fails on many systems even though the resulting python install is perfectly usable for our purposes. The Python spkg-check should be changed to not call the whole python testsuite blindly if it is this finicky. 

[William Stein]

On Jan 17, 2012 10:19 AM, "Volker Braun" <vbrau...@gmail.com> wrote:
>
> I consider it a bug that Python's testsuite fails on many systems even though the resulting python install is perfectly usable for our purposes. The Python spkg-check should be changed to not call the whole python testsuite blindly if it is this finicky. 
>
>

Huge +1

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

[kcrisman]

Hmm, that could be useful, even if not as automated, since it would
mean only typing "make" once, which is the real goal. It wouldn't
lead to an error if I only delete that file (i.e. spkg-install doesn't
have lines about spkg-check in it that would stop the build)?

- kcrisman

[John H Palmieri]

On Tuesday, January 17, 2012 10:19:03 AM UTC-8, Volker Braun wrote:

I consider it a bug that Python's testsuite fails on many systems even though the resulting python install is perfectly usable for our purposes. The Python spkg-check should be changed to not call the whole python testsuite blindly if it is this finicky. 

Absolutely.  With Python 2.7 targeted for inclusion into sage-5.0, we should modify its spkg-check script so it works on all supported platforms.

--
John

[John H Palmieri]

On Tuesday, January 17, 2012 10:45:37 AM UTC-8, kcrisman wrote:

No.  The script sage-spkg (in local/bin) checks whether SAGE_CHECK is set and whether there is a script spkg-check, and if both are true, it runs the script.  So if you delete the script, everything will be fine.

--
John

[kcrisman]

> > Hmm, that could be useful, even if not as automated, since it would
> > mean only typing "make" once, which is the real goal.  It wouldn't
> > lead to an error if I only delete that file (i.e. spkg-install doesn't
> > have lines about spkg-check in it that would stop the build)?
>
> No.  The script sage-spkg (in local/bin) checks whether SAGE_CHECK is set
> and whether there is a script spkg-check, and if both are true, it runs the
> script.  So if you delete the script, everything will be fine.

Okay. If I recall correctly, there are one or two spkgs I've looked
at in the past where the spkg-install does something with SAGE_CHECK
and then does something, though perhaps said spkgs didn't have an spkg-
check file. At any rate, at one point what you said wasn't 100% true,
though it shouldn't impact this issue. Thanks.

[Dan Drake]

Another +1 here.

Is there any way to conveniently extract the failure information from
Python's test suite? Then we could do something like:

expected_failures = set(...)
# do stuff to get the actual failures from Python tests
if not actual_failures.issubset(expected_failures):
# spkg-check says tests failed
else:
# spkg-check exits normally

Dan

--
--- Dan Drake
----- http://mathsci.kaist.ac.kr/~drake
-------

[Dan Drake]

On Tue, 17 Jan 2012 at 10:19AM -0800, Volker Braun wrote:

Fortunately, upon digging in the makefile, I see that it's easy to avoid
the few tests that we know won't work: in spkg-check, just change

make test

to

make EXTRATESTOPTS="-x test_tcl -x test_dbm -x test_gdbm -x test_bsddb" test

Now we just need to know which tests to exclude. On Linux, is it always
those four that fail? On OS X and Solaris, which tests always fail?

[daveloeffler]

I've had consistent failures in "test_distutils" (in all versions of
Sage I've ever tried this with).

David

[Dan Drake]

On Wed, 18 Jan 2012 at 11:17PM +0900, Dan Drake wrote:
> Fortunately, upon digging in the makefile, I see that it's easy to avoid
> the few tests that we know won't work: in spkg-check, just change
>
> make test
>
> to
>
> make EXTRATESTOPTS="-x test_tcl -x test_dbm -x test_gdbm -x test_bsddb" test
>
> Now we just need to know which tests to exclude. On Linux, is it always
> those four that fail? On OS X and Solaris, which tests always fail?

Adding such exclusions to the Python spkg is now
http://trac.sagemath.org/sage_trac/ticket/12321.

I see already that making the list of exclusions might be hard. On
Ubuntu 11.10, the test suite skips a bunch of tests, and says "someone
teach regrtest what tests are expected to be skipped on linux3". So we
need to treat 3.0 Linux kernels differently from 2.6 ones.

(The four tests quoted above are those that fail on
sagenb.kaist.ac.kr -- Ubuntu 10.04.)