Test your Python build
Dr. David Kirkby
are not run. Neither are they run if you build Sage with SAGE_CHECK=yes.
If you want to see if Python builds ok on your system, try this package
http://boxen.math.washington.edu/home/kirkby/revised-patches/python-2.6.4.p9.spkg
It already has positive review at:
http://trac.sagemath.org/sage_trac/ticket/9295
You will need to rebuild python after typing:
$ export SAGE_CHECK=yes
So far, on 3 systems where this has been tested, there are 5 failures on each,
though the 5 failures differ between systems - with one exception
(test_distutils) which seems to fail on all systems.
I intend investigating the failures I get, and perhaps reporting them to the
python bug database
Dave
Alex Ghitza
Hi David,
On Wed, 23 Jun 2010 22:24:44 +0100, "Dr. David Kirkby" <david....@onetel.net> wrote:
> So far, on 3 systems where this has been tested, there are 5 failures on each,
> though the 5 failures differ between systems - with one exception
> (test_distutils) which seems to fail on all systems.
On:
[ghitza@artin ~]$ uname -a
Linux artin 2.6.34-ARCH #1 SMP PREEMPT Sat Jun 19 00:07:49 CEST 2010
x86_64 Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz GenuineIntel GNU/Linux
I got the following:
1 test failed:
test_distutils
Best,
Alex
--
Alex Ghitza -- http://aghitza.org/
Lecturer in Mathematics -- The University of Melbourne -- Australia
Dr. David Kirkby
>
> Hi David,
>
> On Wed, 23 Jun 2010 22:24:44 +0100, "Dr. David Kirkby"<david....@onetel.net> wrote:
>> So far, on 3 systems where this has been tested, there are 5 failures on each,
>> though the 5 failures differ between systems - with one exception
>> (test_distutils) which seems to fail on all systems.
>
> On:
>
> [ghitza@artin ~]$ uname -a
> Linux artin 2.6.34-ARCH #1 SMP PREEMPT Sat Jun 19 00:07:49 CEST 2010
> x86_64 Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz GenuineIntel GNU/Linux
>
> I got the following:
>
> 1 test failed:
> test_distutils
>
>
> Best,
> Alex
>
You are doing better than most. But *everyone* gets that failure. I don't know
how critical that is in Sage, but either the test is broken on that part of
Python is broken in Sage.
Dave
Dr. David Kirkby
Sorry, that did not make a lot of sense.
First, everyone who has tested that package in Sage sees that failure.
Secondly, either
* The distutils test is broken
* Distutils in Python is broken.
* One of the modifications done to Python in Sage has broken disutils.
A few failures are observed on OS X, Solaris and OpenSolaris. Those failures
have always included test_distutils.
If I recall correctly, there are some odd comments on some tickets about
workaround being applied because distutils is broken, though I might be
confusing it with another aspect of Python.
I was rather hoping that Python would get merged, so everyone could test their
Python in Sage, but unfortunately the ticket did not get merged, despite the
positive review. So if anyone wants to check their Python build, they will need
to download
http://boxen.math.washington.edu/home/kirkby/revised-patches/python-2.6.4.p9.spkg
Dave
Alex Ghitza
> On 06/24/10 01:26 AM, Alex Ghitza wrote:
> > On:
> >
> > [ghitza@artin ~]$ uname -a
> > Linux artin 2.6.34-ARCH #1 SMP PREEMPT Sat Jun 19 00:07:49 CEST 2010
> > x86_64 Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz GenuineIntel GNU/Linux
> >
> > I got the following:
> >
> > 1 test failed:
> > test_distutils
>
> how critical that is in Sage, but either the test is broken on that part of
> Python is broken in Sage.
I was curious about this so I downloaded python-2.6.5 and built and
tested it separately. Once again there is one test failure, but it's
not test_distutils, rather test_zlib. This is where my curiosity ends :)
François Bissey
> > On 06/24/10 01:26 AM, Alex Ghitza wrote:
> >> I got the following:
> >>
> >> 1 test failed:
> >> test_distutils
> >>
> >>
> >> Best,
> >> Alex
> >
> > You are doing better than most. But *everyone* gets that failure. I
> > don't know how critical that is in Sage, but either the test is broken
> > on that part of Python is broken in Sage.
> >
> > Dave
>
> Sorry, that did not make a lot of sense.
>
>
> First, everyone who has tested that package in Sage sees that failure.
>
> Secondly, either
>
> * The distutils test is broken
> * Distutils in Python is broken.
> * One of the modifications done to Python in Sage has broken disutils.
>
> A few failures are observed on OS X, Solaris and OpenSolaris. Those
> failures have always included test_distutils.
>
> If I recall correctly, there are some odd comments on some tickets about
> workaround being applied because distutils is broken, though I might be
> confusing it with another aspect of Python.
>
>
From the current python ebuild in gentoo:
# Skip all tests that fail during emerge but pass without emerge:
# (See bug #67970)
local skip_tests="distutils httpservers minidom pyexpat sax tcl"
The bug in question hasn't been touched since 2006 so it is a long standing
issue that no one has looked in a long time.
http://bugs.gentoo.org/show_bug.cgi?id=67970
The strange thing is going manually in the concerned folder and running
make test from there works. Could you check that it is the case in sage too?
One suggestion was some failures were due the environment variable
PYTHON_DONTCOMPILE set to 1 during the build but I am not sure that
would apply in sage.
Francois
Adam Webb
I get two test errors on Scientific Linux 5.1 (32 bit).
test_grp and test_distutils
test_grp is a normal problem on this system.
test test_grp failed -- Traceback (most recent call last):
File "/home/math/sage/spkg/build/python-2.6.4.p9/src/Lib/test/
test_grp.py", line 32, in test_values
e2 = grp.getgrgid(e.gr_gid)
KeyError: 'getgrgid(): gid not found: -2'
This is known problem that getgrgid() does not handle negative
numbers. http://bugs.python.org/issue1465646
test_distutils passes if I use a plain python 2.6.5 tarball. This is
consistent with the problem being in Sage or at least in the
environment used for building packages.
Adam
Dr. David Kirkby
> test_distutils passes if I use a plain python 2.6.5 tarball. This is
> consistent with the problem being in Sage or at least in the
> environment used for building packages.
>
> Adam
>
That's interesting!
So far everyone seems to see this.
* You on Scientific Linux
* Me on OpenSolaris
* Me on Solaris 10
* Alex Ghitza on what I think is probably archlinux though I'm not sure.
* Francois on Gentoo.
So perhaps there is something screwy about how Sage builds Python that causes
this failure.
The Python package has many patches in Sage. Perhaps one of those is faulty.
Dave
François Bissey
> > test_distutils passes if I use a plain python 2.6.5 tarball. This is
> > consistent with the problem being in Sage or at least in the
> > environment used for building packages.
> >
> > Adam
>
> That's interesting!
>
> So far everyone seems to see this.
>
> * You on Scientific Linux
> * Me on OpenSolaris
> * Me on Solaris 10
> * Alex Ghitza on what I think is probably archlinux though I'm not sure.
> * Francois on Gentoo.
>
is known to fail in the "system" package. The snippet I sent is part
of what is in every gentoo system regardless of whether they use
sage or not.
I was pointing out that some python tests are known to fail at the linux
distro level.
Francois
Dr. David Kirkby
I wonder if the problem is fixed in Python 2.6.5, since the test passes for Adam
with 2.6.5.
The problem with Python in Sage is there are so many patches, that is makes it
difficult to update. Many patches are copied over Python files - I don't know
how carefully these changed files have been updated as python has been updated,
but some are years old. The chances someone has screwed up during that time is
quite high.
'hg log' shows 68 entries.
Dave
Adam Webb
On Jun 24, 11:42 pm, "Dr. David Kirkby" <david.kir...@onetel.net>
wrote:
I did a another test and test_distutils also passes for a plain 2.6.4.
In fact, I don't think I have ever seen this one fail. Since my system
Python is only 2.4 I have usually have to compile a newer one.
Normally, I get failures on modules due to missing devel packages.
Naturally failures may vary between distributions. Missing modules may
or may not have an impact on Sage.
I would not rule out any unexpected side-effects of some old patch
though. It is probably a good idea to review the patches to remove
anything that is no longer needed.
Adam
Adam Webb
Dr. David Kirkby
>
>
> On Jun 24, 11:42 pm, "Dr. David Kirkby"<david.kir...@onetel.net>
>> The problem with Python in Sage is there are so many patches, that is makes it
>> difficult to update. Many patches are copied over Python files - I don't know
>> how carefully these changed files have been updated as python has been updated,
>> but some are years old. The chances someone has screwed up during that time is
>> quite high.
>>
>> 'hg log' shows 68 entries.
>>
>> Dave
>
> Hi,
>
> I did a another test and test_distutils also passes for a plain 2.6.4.
> In fact, I don't think I have ever seen this one fail. Since my system
> Python is only 2.4 I have usually have to compile a newer one.
> Normally, I get failures on modules due to missing devel packages.
> Naturally failures may vary between distributions. Missing modules may
> or may not have an impact on Sage.
>
> I would not rule out any unexpected side-effects of some old patch
> though. It is probably a good idea to review the patches to remove
> anything that is no longer needed.
>
> Adam
Hi Adam.
I noticed there was a patch to disutils in Sage, but removing it did not allow
it to build on my system. I would be interested if it does on yours. You should
find it easily in spkg-install - just comment it out.
I do wonder if there should be a different method of applying patches in Sage to
.spkg files. One option is that a checksum is computed of the file being
overwritten, and the patch will only apply if the checksum is the same. That
way, if the upstream source gets updated, the patch can't be applied without
someone manually doing things with it.
Dave
Adam Webb
> > I would not rule out any unexpected side-effects of some old patch
> > though. It is probably a good idea to review the patches to remove
> > anything that is no longer needed.
>
> > Adam
>
> Hi Adam.
>
> I noticed there was a patch to disutils in Sage, but removing it did not allow
> it to build on my system. I would be interested if it does on yours. You should
> find it easily in spkg-install - just comment it out.
>
> I do wonder if there should be a different method of applying patches in Sage to
> .spkg files. One option is that a checksum is computed of the file being
> overwritten, and the patch will only apply if the checksum is the same. That
> way, if the upstream source gets updated, the patch can't be applied without
> someone manually doing things with it.
>
> Dave
systems.
Adam
Peter Jeremy
>On 06/25/10 08:11 AM, Adam Webb wrote:
>> I would not rule out any unexpected side-effects of some old patch
>> though. It is probably a good idea to review the patches to remove
>> anything that is no longer needed.
>
>.spkg files. One option is that a checksum is computed of the file being
>overwritten, and the patch will only apply if the checksum is the same. That
>way, if the upstream source gets updated, the patch can't be applied without
>someone manually doing things with it.
IMHO, the current approach of having 'original' sources under src, then
having patches and patched files in a second directory and 'manually'
copying the patched files over the original files is a sure-fire way to
ensure that things go wrong: Either the "original" file plus patches
does not produce the patched file (eg "original" file has been updated
or the patch has been mis-applied) or the copy script doesn't copy the
updated file to the correct location (either missing it or copying it
to an incorrect location).
My suggestion would be to place the entire spkg under configuration
control and patch files in-place. You then remove the need to
manually copy files around and can rely on hg to warn you of local
patches when importing upgrades.
The downside is a significant increase in the size of the hg
repository. This doesn't affect end-user who only download a binary
distribution. The impact on source distributions could be ameliorated
somewhat by splitting the source distribution into two pieces: Pure
sources (needed to build Sage) and the associated hg files (needed to
create an updated spkg).
--
Peter Jeremy