sage + py.test fails

211 views
Skip to first unread message

Ondrej Certik

unread,
Jul 19, 2008, 4:44:32 PM7/19/08
to sage-...@googlegroups.com
Hi,

when testing sage 3.0.5 using py.test, sage fails to import, because
it's using input (?) stream's write and flush methods:

def __init__(self,stream,fallback):
if not hasattr(stream,'write') or not hasattr(stream,'flush'):
stream = fallback
self.stream = stream
E self._swrite = stream.write
> AttributeError: DontReadFromInput instance has no attribute 'write'

[/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py:54]
________________________________________________________________________________


Full traceback is here:

http://paste.debian.net/11644

Not sure if this is a bug in Sage or py.test, but nevertheless, the
following patch fixes the problem:

--- /tmp/genutils.py 2008-07-19 20:50:56.000000000 +0200
+++ /home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py 2008-07-19
21:00:16.743189357 +0200
@@ -51,8 +51,8 @@
if not hasattr(stream,'write') or not hasattr(stream,'flush'):
stream = fallback
self.stream = stream
- self._swrite = stream.write
- self.flush = stream.flush
+ #self._swrite = stream.write
+ #self.flush = stream.flush

def write(self,data):
try:


Another way to fix it is to put these lines:

import sys
sys.stdin.write = 1
sys.stdin.flush = 1

into a testfile. That's what I am using now. What py.test does is that
it sets sys.stdin to this class:

class DontReadFromInput:
"""Temporary stub class. Ideally when stdin is accessed, the
capturing should be turned off, with possibly all data captured
so far sent to the screen. This should be configurable, though,
because in automated test runs it is better to crash than
hang indefinitely.
"""
def read(self, *args):
raise IOError("reading from stdin while output is captured")
readline = read
readlines = read
__iter__ = read


But I don't understand why ipython wants to write to stdin...

Ondrej

William Stein

unread,
Jul 20, 2008, 7:36:14 AM7/20/08
to sage-...@googlegroups.com
On Sat, Jul 19, 2008 at 10:44 PM, Ondrej Certik <ond...@certik.cz> wrote:
>
> Hi,
>
> when testing sage 3.0.5 using py.test, sage fails to import, because
> it's using input (?) stream's write and flush methods:

I don't know what py.test is, but do you really need to use the IPython
interface to sage with it? Instead of running Sage could you do

sage -python

then put

from sage.all import *

or something? This comment may be completely off, since I
don't know what py.test is. Hey, what's py.test?

--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

Ondrej Certik

unread,
Jul 20, 2008, 9:57:35 AM7/20/08
to sage-...@googlegroups.com
On Sun, Jul 20, 2008 at 1:36 PM, William Stein <wst...@gmail.com> wrote:
>
> On Sat, Jul 19, 2008 at 10:44 PM, Ondrej Certik <ond...@certik.cz> wrote:
>>
>> Hi,
>>
>> when testing sage 3.0.5 using py.test, sage fails to import, because
>> it's using input (?) stream's write and flush methods:
>
> I don't know what py.test is, but do you really need to use the IPython
> interface to sage with it? Instead of running Sage could you do
>
> sage -python
>
> then put
>
> from sage.all import *
>
> or something? This comment may be completely off, since I
> don't know what py.test is. Hey, what's py.test?

http://codespeak.net/py/dist/test.html

it used to be the only good testing framework in python, now there is
also nosetests:

http://www.somethingaboutorange.com/mrl/projects/nose/

that is probably better, and the above problem doesn't arise in there.
But as usual, nosetests and py.test are not equivalent, so we need to
adapt some of our tests first.

sage -python doesn't work:

$ sage -python /usr/bin/py.test sympy/test_external/test_sage.py
Traceback (most recent call last):
File "/usr/bin/py.test", line 9, in <module>
import py
ImportError: No module named py


You would have to set up paths to the system wide modules first. I
want to use system wide python, not Sage's python.

Ondrej

William Stein

unread,
Jul 20, 2008, 10:05:00 AM7/20/08
to sage-...@googlegroups.com

If you're using the system-wide python why does Sage's copy of
IPython have anything to do with anything? I guess I'm basically
asking why you are having to program around a problem with
Ipython in order to do some sort of testing with Sage? For example,
when Sage's doctests run IPython is never involved at all; it's
not even imported.

- William

Ondrej Certik

unread,
Jul 20, 2008, 10:34:32 AM7/20/08
to sage-...@googlegroups.com

I don't want to have anything in common with ipython, but sage invokes
it on import sage.all, as can be checked easily:

ondra@fuji:~/ext/sage$ . local/bin/sage-env
ondra@fuji:~/ext/sage$ python
Python 2.5.2 (r252:60911, Jul 11 2008, 05:28:36)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sage.all
>>>

Then apply this patch:

--- /tmp/genutils.py 2008-07-20 16:33:15.000000000 +0200
+++ local/lib/python2.5/site-packages/IPython/genutils.py 2008-07-20
16:33:26.553433732 +0200
@@ -54,6 +54,7 @@


if not hasattr(stream,'write') or not hasattr(stream,'flush'):
stream = fallback
self.stream = stream

+ stop
self._swrite = stream.write
self.flush = stream.flush

and:

ondra@fuji:~/ext/sage$ python
Python 2.5.2 (r252:60911, Jul 11 2008, 05:28:36)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sage.all


Traceback (most recent call last):

File "<stdin>", line 1, in <module>
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/sage/all.py",
line 58, in <module>
from sage.misc.all import * # takes a while
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/sage/misc/all.py",
line 15, in <module>
from sage_timeit_class import timeit
File "sage_timeit_class.pyx", line 3, in sage.misc.sage_timeit_class
(sage/misc/sage_timeit_class.c:485)
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/sage/misc/sage_timeit.py",
line 12, in <module>
import timeit as timeit_, time, math, preparser, interpreter
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/sage/misc/interpreter.py",
line 108, in <module>
from IPython.iplib import InteractiveShell
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/__init__.py",
line 57, in <module>
__import__(name,glob,loc,[])
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/ipstruct.py",
line 22, in <module>
from IPython.genutils import list2dict2
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py",
line 95, in <module>
Term = IOTerm()
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py",
line 90, in __init__
self.cin = IOStream(cin,sys.stdin)
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py",
line 57, in __init__
stop
NameError: global name 'stop' is not defined
>>>

Ondrej

William Stein

unread,
Jul 20, 2008, 10:48:21 AM7/20/08
to sage-...@googlegroups.com
On Sun, Jul 20, 2008 at 4:34 PM, Ondrej Certik <ond...@certik.cz> wrote:
>> If you're using the system-wide python why does Sage's copy of
>> IPython have anything to do with anything? I guess I'm basically
>> asking why you are having to program around a problem with
>> Ipython in order to do some sort of testing with Sage? For example,
>> when Sage's doctests run IPython is never involved at all; it's
>> not even imported.
>
> I don't want to have anything in common with ipython, but sage invokes
> it on import sage.all, as can be checked easily:

Wow, that sucks. Thanks for pointing this out!! It is a major bug which
i'll fix asap:

http://trac.sagemath.org/sage_trac/ticket/3685

Thanks again -- I really appreciate this.

William

--

Ondrej Certik

unread,
Jul 20, 2008, 11:20:47 AM7/20/08
to sage-...@googlegroups.com
On Sun, Jul 20, 2008 at 4:48 PM, William Stein <wst...@gmail.com> wrote:
>
> On Sun, Jul 20, 2008 at 4:34 PM, Ondrej Certik <ond...@certik.cz> wrote:
>>> If you're using the system-wide python why does Sage's copy of
>>> IPython have anything to do with anything? I guess I'm basically
>>> asking why you are having to program around a problem with
>>> Ipython in order to do some sort of testing with Sage? For example,
>>> when Sage's doctests run IPython is never involved at all; it's
>>> not even imported.
>>
>> I don't want to have anything in common with ipython, but sage invokes
>> it on import sage.all, as can be checked easily:
>
> Wow, that sucks. Thanks for pointing this out!! It is a major bug which
> i'll fix asap:
>
> http://trac.sagemath.org/sage_trac/ticket/3685
>
> Thanks again -- I really appreciate this.

Thanks too.

BTW, here is another creepy thing, that maybe is a bug in Sage:

$ gedit
gedit: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
$

I was just about to report this as a grave bug in Debian, but before I did:

$ ldd /usr/lib/libxml2.so.2
linux-gate.so.1 => (0xb7f92000)
libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7e4e000)
libz.so.1 => /home/ondra/ext/sage/local/lib/libz.so.1 (0xb7e38000)
libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7e11000)
libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7cb6000)
/lib/ld-linux.so.2 (0xb7f93000)


so I just removed the line:

export LD_LIBRARY_PATH=/home/ondra/ext/sage/local/lib

from my bashrc and all works again. So this is not the way how to
setup sage. It's actually quite annoying, that you can use either Sage
or gedit, but not both in the same terminal. Ok, you may say forget
about systemwide python, just use Sage as it is, take it or leave it.
Ok, first, it's not the way I'd like to work, but ok, but it won't
allow me to use gedit either... Try this:

sage: os.system("xclock")
0

this works anc xclock comes up, but this doesn't:

sage: os.system("gedit")

** (gedit:26452): WARNING **: Error initializing Python interpreter:
could not import pygtk.

** (gedit:26452): WARNING **: Please check the installation of all the
Python related packages required by gedit and try again.

** (gedit:26452): WARNING **: Cannot load Python plugin 'Python
Console' since gedit was not able to initialize the Python
interpreter.
gedit: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
32512


I don't know, but I think Sage should not be messing with my system.

So this makes me think that maybe exporting the variables so that they
work in the whole terminal (thus allowing me to use systemwide python)
is not a good thing, unless there is a robust way to fix the above
problems. I'll try to investiage the other way round then, e.g.
setting up paths in sage, so that it imports my systemwide library and
the current revision of sympy, that I want to test. But I don't like
this at all, because this makes Sage a full blown application, not a
library, that can be reused in other projects.

Ondrej

Ondrej Certik

unread,
Jul 20, 2008, 11:32:00 AM7/20/08
to sage-...@googlegroups.com

There must be a way to get this fixed, so that Sage can be used as a
library and it can use it's own compiled libs, but it will not force
the rest of the system to use them. Or there isn't?

Ondrej

William Stein

unread,
Jul 20, 2008, 11:45:32 AM7/20/08
to sage-...@googlegroups.com
> There must be a way to get this fixed, so that Sage can be used as a
> library and it can use it's own compiled libs, but it will not force
> the rest of the system to use them. Or there isn't?

There pretty much isn't.

However you can maybe do:

sage: os.system('unset LD_LIBRARY_PATH; gedit')

Alternatively, we have a script sage-native-execute

sage-native-execute gedit

that is in SAGE_ROOT/local/bin/
that Sage itself uses whenever sage library code calls
system-wide binaries.

-- William

William Stein

unread,
Jul 20, 2008, 11:48:29 AM7/20/08
to sage-...@googlegroups.com
On Sun, Jul 20, 2008 at 4:48 PM, William Stein <wst...@gmail.com> wrote:
> On Sun, Jul 20, 2008 at 4:34 PM, Ondrej Certik <ond...@certik.cz> wrote:
>>> If you're using the system-wide python why does Sage's copy of
>>> IPython have anything to do with anything? I guess I'm basically
>>> asking why you are having to program around a problem with
>>> Ipython in order to do some sort of testing with Sage? For example,
>>> when Sage's doctests run IPython is never involved at all; it's
>>> not even imported.
>>
>> I don't want to have anything in common with ipython, but sage invokes
>> it on import sage.all, as can be checked easily:
>
> Wow, that sucks. Thanks for pointing this out!! It is a major bug which
> i'll fix asap:
>
> http://trac.sagemath.org/sage_trac/ticket/3685
>
> Thanks again -- I really appreciate this.
>

There is now a patch up at
http://trac.sagemath.org/sage_trac/ticket/3685
that fixes the above-mentioned problem.

William

Ondrej Certik

unread,
Jul 20, 2008, 12:25:33 PM7/20/08
to sage-...@googlegroups.com

This script just sets the LD_LIBRARY_PATH to "" (in my case). Indeed, the

sage: os.system('unset LD_LIBRARY_PATH; gedit')

works, but because the PATHs are set to use sage python, other
programs will not run anyway:

sage: os.system('unset LD_LIBRARY_PATH; mayavi2')


Traceback (most recent call last):

File "/usr/bin/mayavi2", line 3, in <module>
from enthought.mayavi.scripts import mayavi2
ImportError: No module named mayavi.scripts
256


Hm, Sage just needs LD_LIBRARY_PATH and PATH set to something, while
the rest of the system to something else.

Well, PATHs could be fixed somehow. The problem is LD_LIBRARY_PATH --
but I think there is something like chrpath, that could fix the
problem.
E.g. maybe it's possible to write a script that sets this in Sage
automatically on the first run. I just believe there must be a way to
get this fixed.

The other option is to use Sage debian packages.

> There is now a patch up at
> http://trac.sagemath.org/sage_trac/ticket/3685
> that fixes the above-mentioned problem.

Thanks for fixing this!

Ondrej

mabshoff

unread,
Jul 20, 2008, 12:38:42 PM7/20/08
to sage-devel
On Jul 20, 8:20 am, "Ondrej Certik" <ond...@certik.cz> wrote:
> On Sun, Jul 20, 2008 at 4:48 PM, William Stein <wst...@gmail.com> wrote:

Hi,

<SNIP>

> BTW, here is another creepy thing, that maybe is a bug in Sage:
>
> $ gedit
> gedit: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
> $
>
> I was just about to report this as a grave bug in Debian, but before I did:
>
> $ ldd /usr/lib/libxml2.so.2
>         linux-gate.so.1 =>  (0xb7f92000)
>         libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7e4e000)
>         libz.so.1 => /home/ondra/ext/sage/local/lib/libz.so.1 (0xb7e38000)
>         libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7e11000)
>         libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7cb6000)
>         /lib/ld-linux.so.2 (0xb7f93000)
>
> so I just removed the line:
>
> export LD_LIBRARY_PATH=/home/ondra/ext/sage/local/lib
>
> from my bashrc and all works again. So this is not the way how to
> setup sage. It's actually quite annoying, that you can use either Sage
> or gedit, but not both in the same terminal. Ok, you may say forget
> about systemwide python, just use Sage as it is, take it or leave it.

I am please to say: I told you so. Running Sage with the system python
introduces all kinds of the above crap. And as William pointed out it
is not a bug and we will not fix it since we cannot fix it. There are
myriads of distros out there and no way to make Sage+system Python
work since the libraries they ship are in the end mutually
incompatible. This is one giant quagmire and there is no way out of it
besides using the Debian packages that Tim has been working on.

> Ok, first, it's not the way I'd like to work, but ok, but it won't
> allow me to use gedit either... Try this:
>
> sage: os.system("xclock")
> 0
>
> this works anc xclock comes up, but this doesn't:
>
> sage: os.system("gedit")
>
> ** (gedit:26452): WARNING **: Error initializing Python interpreter:
> could not import pygtk.
>
> ** (gedit:26452): WARNING **: Please check the installation of all the
> Python related packages required by gedit and try again.
>
> ** (gedit:26452): WARNING **: Cannot load Python plugin 'Python
> Console' since gedit was not able to initialize the Python
> interpreter.
> gedit: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
> 32512
>
> I don't know, but I think Sage should not be messing with my system.

Sage is not messing with your system, you *choose* to do things with
LD_LIBRARY_PATH and it blew up in your face. See another "I told you
so" coming? :p

When you launch an application from "proper" Sage that is not
$SAGE_LOCAL/bin it will use the original LD_LIBRARY_PATH. In case
python based applications we might also have to set PATH back to its
original so that the system Python is picked up, but then all the Sage
binaries in $SAGE?LOCAL/bin are not available, so I do not see any
solution that will make everybody happy.

> So this makes me think that maybe exporting the variables so that they
> work in the whole terminal (thus allowing me to use systemwide python)
> is not a good thing, unless there is a robust way to fix the above
> problems.

I think the problem is unfixable. Either way you chose to fix the
issue I can break it. Believe me: I have thought about the issue for a
while :)

> I'll try to investiage the other way round then, e.g.
> setting up paths in sage, so that it imports my systemwide library and
> the current revision of sympy, that I want to test. But I don't like
> this at all, because this makes Sage a full blown application, not a
> library, that can be reused in other projects.

Sage is and never has been meant to be a library in the sense that
Sympy is. In itself Sage is very complex and we want to run on
operating systems besides Debian/Linux, so I see no reason to cater to
any specific distribution of Linux and "just use the system zlib" will
not work everywhere. Sage is self contained for a reason and if we
relied on more components by the system Sage would not work very well.
Just think of the rather large number of combinations when only taken
ten of Sage's components and using the system's instead. I have no
desire to debug such a mess.

You can fix the above issue by skipping the build of zlib by fixing
deps in $SAGE_ROOT/spkg/standard. But that will require rebuilding
Sage from source unless you are willing to remove libz* and the
headers and then rebuild all components of Sage that link against
libz. So a rebuild from scratch in another directory will be quicker
for you since it will take you longer to figure out what to do since
you are not as familiar with Sage's components as I am :)

But even then it will only be a question of time until something else
blows up. That is why I strictly refuse to support Sage being used in
that way in the first place.

> Ondrej

Cheers,

Michael

Ondrej Certik

unread,
Jul 20, 2008, 2:20:59 PM7/20/08
to sage-...@googlegroups.com

Sage should use its own versions of libraries for the reasons you
stated. One exception could be Python,
e.g. there could be an easy way to compile Sage with the systemwide
python, because (imho) Python doesn't change that much, so it could
work in most cases. It works even now.

As chatted on IRC, the LD_LIBRARY_PATH problem *might* get fixed using:

http://packages.debian.org/sid/chrpath

And the other PATH stuff are fixable too.

Ondrej

mabshoff

unread,
Jul 20, 2008, 4:02:56 PM7/20/08
to sage-devel


On Jul 20, 11:20 am, "Ondrej Certik" <ond...@certik.cz> wrote:
> On Sun, Jul 20, 2008 at 6:38 PM, mabshoff <mabsh...@googlemail.com> wrote:

<SNIP>

Hi Ondrej,

> > But even then it will only be a question of time until something else
> > blows up. That is why I strictly refuse to support Sage being used in
> > that way in the first place.
>
> Sage should use its own versions of libraries for the reasons you
> stated. One exception could be Python,
> e.g. there could be an easy way to compile Sage with the systemwide
> python, because (imho) Python doesn't change that much, so it could
> work in most cases. It works even now.

Nope, it does not work for example on OSX. With a python 2.5.2 on
Linux x86 and x86 it should mostly work, but on Linux Itanium it does
not work since in the official 2.5.2 readline is busted. On Solaris
and OSX there is also the multilib issue, i.e. 32 vs. 64 bit are
supported on the same machine. So I disagree that it mostly works and
I consider it a bad, bad idea to even give the user some simple option
to make Sage build with the system python. It is trivial to do if you
know your way around the Sage build system, but that actually means
that one can likely fix issues oneself like in your case. Everybody
else just has to play by the rules.

And I won't post the obvious one line diff to make Sage with system
python at build time work. If you need to ask you shouldn't play with
something that could cause endless trouble :p

> As chatted on IRC, the LD_LIBRARY_PATH problem *might* get fixed using:
>
> http://packages.debian.org/sid/chrpath

Yes, this certainly offers possibilities.

> And the other PATH stuff are fixable too.
>
> Ondrej

Cheers,

Michael

Ondrej Certik

unread,
Jul 20, 2008, 4:22:56 PM7/20/08
to sage-...@googlegroups.com
On Sun, Jul 20, 2008 at 10:02 PM, mabshoff <mabs...@googlemail.com> wrote:
>
>
>
> On Jul 20, 11:20 am, "Ondrej Certik" <ond...@certik.cz> wrote:
>> On Sun, Jul 20, 2008 at 6:38 PM, mabshoff <mabsh...@googlemail.com> wrote:
>
> <SNIP>
>
> Hi Ondrej,
>
>> > But even then it will only be a question of time until something else
>> > blows up. That is why I strictly refuse to support Sage being used in
>> > that way in the first place.
>>
>> Sage should use its own versions of libraries for the reasons you
>> stated. One exception could be Python,
>> e.g. there could be an easy way to compile Sage with the systemwide
>> python, because (imho) Python doesn't change that much, so it could
>> work in most cases. It works even now.
>
> Nope, it does not work for example on OSX. With a python 2.5.2 on

I don't understand the OSX problems about universal/non universal
python (ucs2 vs ucs4 surely can be fixed).

> Linux x86 and x86 it should mostly work, but on Linux Itanium it does
> not work since in the official 2.5.2 readline is busted. On Solaris

That really sucks. I hope upstream fixes that.

> and OSX there is also the multilib issue, i.e. 32 vs. 64 bit are
> supported on the same machine. So I disagree that it mostly works and
> I consider it a bad, bad idea to even give the user some simple option
> to make Sage build with the system python. It is trivial to do if you
> know your way around the Sage build system, but that actually means
> that one can likely fix issues oneself like in your case. Everybody
> else just has to play by the rules.

Well, if the situation is that the system wide python is so different
on different machines that it makes Sage fail, then it sucks. Imho.

>
> And I won't post the obvious one line diff to make Sage with system
> python at build time work. If you need to ask you shouldn't play with
> something that could cause endless trouble :p

I understand your point as the release manager, but from the users
point of view, if every program distributed all the libraries (python,
fortran, libxml, and all the other stuff that ships with sage), and it
would be incompatible with the other installations, as currently Sage
is incompatible with systemwide python as you stressed several times,
then it's bad, because I cannot mix Sage with other components on my
system. So it's Sage and the rest of the world.

>> As chatted on IRC, the LD_LIBRARY_PATH problem *might* get fixed using:
>>
>> http://packages.debian.org/sid/chrpath
>
> Yes, this certainly offers possibilities.

Fortunately.

Ondrej

mabshoff

unread,
Jul 20, 2008, 5:23:19 PM7/20/08
to sage-devel


On Jul 20, 1:22 pm, "Ondrej Certik" <ond...@certik.cz> wrote:

<SNIP>

Hi Ondrej,

> > Nope, it does not work for example on OSX. With a python 2.5.2 on
>
> I don't understand the OSX problems about universal/non universal
> python (ucs2 vs ucs4 surely can be fixed).
>
> > Linux x86 and x86 it should mostly work, but on Linux Itanium it does
> > not work since in the official 2.5.2 readline is busted. On Solaris
>
> That really sucks. I hope upstream fixes that.

I am not even sure the Python people know. IIRC William told me that
at lease SLES on Itanium just disabled building the readline extension
for Python.

> > and OSX there is also the multilib issue, i.e. 32 vs. 64 bit are
> > supported on the same machine. So I disagree that it mostly works and
> > I consider it a bad, bad idea to even give the user some simple option
> > to make Sage build with the system python. It is trivial to do if you
> > know your way around the Sage build system, but that actually means
> > that one can likely fix issues oneself like in your case. Everybody
> > else just has to play by the rules.
>
> Well, if the situation is that the system wide python is so different
> on different machines that it makes Sage fail, then it sucks. Imho.

It won't necessarily fail, but there are many possibilities for
potentially very subtle bugs. And since I spend an (unresonable)
amount of time chasing down pretty much every issue reported I would
prefer that the number of variables does not increase. In addition it
is not my fault that Apple decided that a universal Python is the way
to go. Once we switch to a pure 64 bit Python on OSX Apple's python
won't work at all any more anyway.

And this problem on OSX is not some hypothetical issue: I spend two
hours a couple weeks ago debugging a python import module failure on
OSX where universal libraries played a role and the linker happily
linked together an x86 extension and a ppc library. The import of the
module failed and it was far from clear why. And if it takes me two
hours to figure out you can guess what I would assume what would
happen to most people. And in that case the situation was clear cut
and I had shell access to the box in question. Now imagine debugging
that problem by email :)

> > And I won't post the obvious one line diff to make Sage with system
> > python at build time work. If you need to ask you shouldn't play with
> > something that could cause endless trouble :p
>
> I understand your point as the release manager, but from the users
> point of view, if every program distributed all the libraries (python,
> fortran, libxml, and all the other stuff that ships with sage), and it
> would be incompatible with the other installations, as currently Sage
> is incompatible with systemwide python as you stressed several times,
> then it's bad, because I cannot mix Sage with other components on my
> system. So it's Sage and the rest of the world.

Well, Sage works because it is self contained and it is only possible
to work because it is self contained. We can fix bugs rapidly because
we nearly control 100% of the components and its dependencies of Sage.
No one ever promised that Sage would play well with your system and
installing Python packages into a Sage install is trivial. You can
trivially use your system's Python, but I have no intentions of making
it easy on you because other people will see the email and will shoot
themselves in the foot. Even once Sage is in Debian proper I will not
consider anything a bug unless it can be reproduced with a vanilla
Sage build from sources. There is only one official Sage release and
it will not be the one in any distribution. I seriously doubt that any
distribution can keep up with the official Sage.

One big issue I see: What are you going to do when the Sage project
updates to Python 2.6 and then to 3.0 and Debian does not keep pace or
the other way around when say Debian moved to Python 3 before we do.
Then you just FUBARed your current Sage install and people will blame
Sage and not their own stupidity. The same applies to the issue with
symbols in libz that you initially reported. It is trivial to
understand why it happens but again I suspect that many people will
blame Sage and not even report it to the mailing list. And those are
the people who might then go back to some other CAS and have the
impression that Sage sucks. So in the end I see it like this: A minor
inconvenience you can fix by thinking about it for a second vs. a
potentially huge fuckup for loads of n00bs - I don't have to think
about the choices here :)

> >> As chatted on IRC, the LD_LIBRARY_PATH problem *might* get fixed using:
>
> >>http://packages.debian.org/sid/chrpath
>
> > Yes, this certainly offers possibilities.
>
> Fortunately.

Yes, I think we can fix that via some script to make you happy :)

> Ondrej

Cheers,

Michael

William Stein

unread,
Jul 20, 2008, 5:45:01 PM7/20/08
to sage-...@googlegroups.com
On Sun, Jul 20, 2008 at 11:23 PM, mabshoff <mabs...@googlemail.com> wrote:
>
>
>
> On Jul 20, 1:22 pm, "Ondrej Certik" <ond...@certik.cz> wrote:
>
> <SNIP>
>
> Hi Ondrej,
>
>> > Nope, it does not work for example on OSX. With a python 2.5.2 on
>>
>> I don't understand the OSX problems about universal/non universal
>> python (ucs2 vs ucs4 surely can be fixed).
>>
>> > Linux x86 and x86 it should mostly work, but on Linux Itanium it does
>> > not work since in the official 2.5.2 readline is busted. On Solaris
>>
>> That really sucks. I hope upstream fixes that.
>
> I am not even sure the Python people know. IIRC William told me that
> at lease SLES on Itanium just disabled building the readline extension
> for Python.

We officially reported the bug to them via their tracker.
Then one Sage developer worked for a long time and
heroically found a fix and reported it too. I think it maybe
just takes a long time to filter through the system back
to the *deployed* RHEL and SLES linux boxes all over
the place. I.e., using the system-wide Python by
default just isn't an option. And of course a lot of users
want to install Sage without having to be root, and they
don't know anything about Python or installing it themselves.

Also let me add that the goal of Sage is to provide a viable alternative
to the Ma's and that all of those programs are self-contained. Our design
decisions are very much motivated by the Sage mission statement. We
have to be really focused on that goal, and consequently *not* worry about
certain other very worthy goals like nicely integrating as a library. That's
not to say that it isn't *wonderful* that some people like Tim Abbot are
doing an amazing job on the goal of integrating as a library. Speaking
of which, I think debian-sage is the perfect solution for you Ondrej.
Of course you know all about it since you're very involved in that effort.
(I get the sense it will be ready soon too.)

>> > And I won't post the obvious one line diff to make Sage with system
>> > python at build time work. If you need to ask you shouldn't play with
>> > something that could cause endless trouble :p
>>
>> I understand your point as the release manager, but from the users
>> point of view, if every program distributed all the libraries (python,
>> fortran, libxml, and all the other stuff that ships with sage), and it
>> would be incompatible with the other installations, as currently Sage
>> is incompatible with systemwide python as you stressed several times,
>> then it's bad, because I cannot mix Sage with other components on my
>> system. So it's Sage and the rest of the world.
>
> Well, Sage works because it is self contained and it is only possible
> to work because it is self contained. We can fix bugs rapidly because
> we nearly control 100% of the components and its dependencies of Sage.
> No one ever promised that Sage would play well with your system and
> installing Python packages into a Sage install is trivial. You can
> trivially use your system's Python, but I have no intentions of making
> it easy on you because other people will see the email and will shoot
> themselves in the foot. Even once Sage is in Debian proper I will not
> consider anything a bug unless it can be reproduced with a vanilla
> Sage build from sources. There is only one official Sage release and
> it will not be the one in any distribution. I seriously doubt that any
> distribution can keep up with the official Sage.

That said, it will be very good for there to be a Sage in Debian, and
I'm really looking forward to the day that happens. I know because
before for many many users I was personally a Linux user
for which the world of software was equal to the world of
packages available in rpm or deb's. I can understand that perspective,
and want to make using Sage easy for such people.

> One big issue I see: What are you going to do when the Sage project
> updates to Python 2.6 and then to 3.0 and Debian does not keep pace or
> the other way around when say Debian moved to Python 3 before we do.
> Then you just FUBARed your current Sage install and people will blame
> Sage and not their own stupidity. The same applies to the issue with
> symbols in libz that you initially reported. It is trivial to
> understand why it happens but again I suspect that many people will
> blame Sage and not even report it to the mailing list. And those are
> the people who might then go back to some other CAS and have the
> impression that Sage sucks. So in the end I see it like this: A minor
> inconvenience you can fix by thinking about it for a second vs. a
> potentially huge fuckup for loads of n00bs - I don't have to think
> about the choices here :)

I strongly agree with this. And it is people exactly not getting
this point that is I think a big problem that has really held back
open source math software from providing a genuine alternative
to the Ma's for power users like me or your typical calculus
student.


>> >> As chatted on IRC, the LD_LIBRARY_PATH problem *might* get fixed using:
>>
>> >>http://packages.debian.org/sid/chrpath
>>
>> > Yes, this certainly offers possibilities.
>>
>> Fortunately.
>
> Yes, I think we can fix that via some script to make you happy :)
>
>> Ondrej
>
> Cheers,
>
> Michael
> >
>

--

Ondrej Certik

unread,
Jul 20, 2008, 7:11:59 PM7/20/08
to sage-...@googlegroups.com
> We officially reported the bug to them via their tracker.
> Then one Sage developer worked for a long time and
> heroically found a fix and reported it too. I think it maybe
> just takes a long time to filter through the system back
> to the *deployed* RHEL and SLES linux boxes all over
> the place. I.e., using the system-wide Python by
> default just isn't an option. And of course a lot of users
> want to install Sage without having to be root, and they
> don't know anything about Python or installing it themselves.

I see.

Unfortunately not lately, but I at least created the list and wiki
pages. So all credit goes to Tim. He has done a fenomenal job.

Yes, That should fix the problem. Nevertheless I still want to find
ways to easily test newest Sage and newest SymPy in some canonical way
without messing up with paths and shooting myself in the foot, to use
mabshoff's words.

users -> years? How many years were you using linux and why did you
switch to Mac OS X?
I am thinking of buying a macbook, but installing Debian on it. :)

> for which the world of software was equal to the world of
> packages available in rpm or deb's. I can understand that perspective,
> and want to make using Sage easy for such people.

I understand the other perspective too, and actually, I may endup
teaching some undergrad classes in a year or two, so I will use Sage
package, not Sage debian package, simply because the people will be
using windows, not Debian. Or they will use ubuntu, but still it's
easy to just download and untar. Even though unfortunately, on my
laptop it takes many minutes, maybe up to 15 min together
(download+untarring+first sage run).

>
>> One big issue I see: What are you going to do when the Sage project
>> updates to Python 2.6 and then to 3.0 and Debian does not keep pace or
>> the other way around when say Debian moved to Python 3 before we do.
>> Then you just FUBARed your current Sage install and people will blame
>> Sage and not their own stupidity. The same applies to the issue with
>> symbols in libz that you initially reported. It is trivial to
>> understand why it happens but again I suspect that many people will
>> blame Sage and not even report it to the mailing list. And those are
>> the people who might then go back to some other CAS and have the
>> impression that Sage sucks. So in the end I see it like this: A minor
>> inconvenience you can fix by thinking about it for a second vs. a
>> potentially huge fuckup for loads of n00bs - I don't have to think
>> about the choices here :)
>
> I strongly agree with this. And it is people exactly not getting
> this point that is I think a big problem that has really held back
> open source math software from providing a genuine alternative
> to the Ma's for power users like me or your typical calculus
> student.

I strongly agree with this too. Some people say I see things too much
in black and white (was it you?:) and they are right. I think that if
you download a program and untar it, it needs to just work without
setting anything. Maybe doing one "make", if there is no binary. If it
fails, it's a showstopper and I think it's a perfectly valid reason to
go to some other CAS that just works.

Nevertheless I also think that a good program first needs to satisfy
the above, but if it does and there are resources then it should be
also possible to tweak it in a way to do things that I want. If you
know what I mean. So I will continue crafting that little script that
will make me happy.

Ondrej

William Stein

unread,
Jul 21, 2008, 1:14:12 AM7/21/08
to sage-...@googlegroups.com
>> That said, it will be very good for there to be a Sage in Debian, and
>> I'm really looking forward to the day that happens. I know because
>> before for many many users I was personally a Linux user
>
> users -> years?

yes

> How many years were you using linux and why did you
> switch to Mac OS X?

I use Linux everyday, e.g., on sage.math, and have since 1991.
I just use Mac OS X for my laptop, since I find Linux on Laptops a bit painful.
I really meant more using Linux as more of a button pushing
end user than as somebody who would put a lot of work into building
software from source.

OK from me. Good luck avoiding the wrath of mabshoff :-)

-- William

Reply all
Reply to author
Forward
0 new messages