when testing sage 3.0.5 using py.test, sage fails to import, because
it's using input (?) stream's write and flush methods:
def __init__(self,stream,fallback):
if not hasattr(stream,'write') or not hasattr(stream,'flush'):
stream = fallback
self.stream = stream
E self._swrite = stream.write
> AttributeError: DontReadFromInput instance has no attribute 'write'
[/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py:54]
________________________________________________________________________________
Full traceback is here:
Not sure if this is a bug in Sage or py.test, but nevertheless, the
following patch fixes the problem:
--- /tmp/genutils.py 2008-07-19 20:50:56.000000000 +0200
+++ /home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py 2008-07-19
21:00:16.743189357 +0200
@@ -51,8 +51,8 @@
if not hasattr(stream,'write') or not hasattr(stream,'flush'):
stream = fallback
self.stream = stream
- self._swrite = stream.write
- self.flush = stream.flush
+ #self._swrite = stream.write
+ #self.flush = stream.flush
def write(self,data):
try:
Another way to fix it is to put these lines:
import sys
sys.stdin.write = 1
sys.stdin.flush = 1
into a testfile. That's what I am using now. What py.test does is that
it sets sys.stdin to this class:
class DontReadFromInput:
"""Temporary stub class. Ideally when stdin is accessed, the
capturing should be turned off, with possibly all data captured
so far sent to the screen. This should be configurable, though,
because in automated test runs it is better to crash than
hang indefinitely.
"""
def read(self, *args):
raise IOError("reading from stdin while output is captured")
readline = read
readlines = read
__iter__ = read
But I don't understand why ipython wants to write to stdin...
Ondrej
I don't know what py.test is, but do you really need to use the IPython
interface to sage with it? Instead of running Sage could you do
sage -python
then put
from sage.all import *
or something? This comment may be completely off, since I
don't know what py.test is. Hey, what's py.test?
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
http://codespeak.net/py/dist/test.html
it used to be the only good testing framework in python, now there is
also nosetests:
http://www.somethingaboutorange.com/mrl/projects/nose/
that is probably better, and the above problem doesn't arise in there.
But as usual, nosetests and py.test are not equivalent, so we need to
adapt some of our tests first.
sage -python doesn't work:
$ sage -python /usr/bin/py.test sympy/test_external/test_sage.py
Traceback (most recent call last):
File "/usr/bin/py.test", line 9, in <module>
import py
ImportError: No module named py
You would have to set up paths to the system wide modules first. I
want to use system wide python, not Sage's python.
Ondrej
If you're using the system-wide python why does Sage's copy of
IPython have anything to do with anything? I guess I'm basically
asking why you are having to program around a problem with
Ipython in order to do some sort of testing with Sage? For example,
when Sage's doctests run IPython is never involved at all; it's
not even imported.
- William
I don't want to have anything in common with ipython, but sage invokes
it on import sage.all, as can be checked easily:
ondra@fuji:~/ext/sage$ . local/bin/sage-env
ondra@fuji:~/ext/sage$ python
Python 2.5.2 (r252:60911, Jul 11 2008, 05:28:36)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sage.all
>>>
Then apply this patch:
--- /tmp/genutils.py 2008-07-20 16:33:15.000000000 +0200
+++ local/lib/python2.5/site-packages/IPython/genutils.py 2008-07-20
16:33:26.553433732 +0200
@@ -54,6 +54,7 @@
if not hasattr(stream,'write') or not hasattr(stream,'flush'):
stream = fallback
self.stream = stream
+ stop
self._swrite = stream.write
self.flush = stream.flush
and:
ondra@fuji:~/ext/sage$ python
Python 2.5.2 (r252:60911, Jul 11 2008, 05:28:36)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sage.all
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/sage/all.py",
line 58, in <module>
from sage.misc.all import * # takes a while
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/sage/misc/all.py",
line 15, in <module>
from sage_timeit_class import timeit
File "sage_timeit_class.pyx", line 3, in sage.misc.sage_timeit_class
(sage/misc/sage_timeit_class.c:485)
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/sage/misc/sage_timeit.py",
line 12, in <module>
import timeit as timeit_, time, math, preparser, interpreter
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/sage/misc/interpreter.py",
line 108, in <module>
from IPython.iplib import InteractiveShell
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/__init__.py",
line 57, in <module>
__import__(name,glob,loc,[])
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/ipstruct.py",
line 22, in <module>
from IPython.genutils import list2dict2
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py",
line 95, in <module>
Term = IOTerm()
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py",
line 90, in __init__
self.cin = IOStream(cin,sys.stdin)
File "/home/ondra/ext/sage/local/lib/python2.5/site-packages/IPython/genutils.py",
line 57, in __init__
stop
NameError: global name 'stop' is not defined
>>>
Ondrej
Wow, that sucks. Thanks for pointing this out!! It is a major bug which
i'll fix asap:
http://trac.sagemath.org/sage_trac/ticket/3685
Thanks again -- I really appreciate this.
William
--
Thanks too.
BTW, here is another creepy thing, that maybe is a bug in Sage:
$ gedit
gedit: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
$
I was just about to report this as a grave bug in Debian, but before I did:
$ ldd /usr/lib/libxml2.so.2
linux-gate.so.1 => (0xb7f92000)
libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7e4e000)
libz.so.1 => /home/ondra/ext/sage/local/lib/libz.so.1 (0xb7e38000)
libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7e11000)
libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7cb6000)
/lib/ld-linux.so.2 (0xb7f93000)
so I just removed the line:
export LD_LIBRARY_PATH=/home/ondra/ext/sage/local/lib
from my bashrc and all works again. So this is not the way how to
setup sage. It's actually quite annoying, that you can use either Sage
or gedit, but not both in the same terminal. Ok, you may say forget
about systemwide python, just use Sage as it is, take it or leave it.
Ok, first, it's not the way I'd like to work, but ok, but it won't
allow me to use gedit either... Try this:
sage: os.system("xclock")
0
this works anc xclock comes up, but this doesn't:
sage: os.system("gedit")
** (gedit:26452): WARNING **: Error initializing Python interpreter:
could not import pygtk.
** (gedit:26452): WARNING **: Please check the installation of all the
Python related packages required by gedit and try again.
** (gedit:26452): WARNING **: Cannot load Python plugin 'Python
Console' since gedit was not able to initialize the Python
interpreter.
gedit: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64
32512
I don't know, but I think Sage should not be messing with my system.
So this makes me think that maybe exporting the variables so that they
work in the whole terminal (thus allowing me to use systemwide python)
is not a good thing, unless there is a robust way to fix the above
problems. I'll try to investiage the other way round then, e.g.
setting up paths in sage, so that it imports my systemwide library and
the current revision of sympy, that I want to test. But I don't like
this at all, because this makes Sage a full blown application, not a
library, that can be reused in other projects.
Ondrej
There must be a way to get this fixed, so that Sage can be used as a
library and it can use it's own compiled libs, but it will not force
the rest of the system to use them. Or there isn't?
Ondrej
There pretty much isn't.
However you can maybe do:
sage: os.system('unset LD_LIBRARY_PATH; gedit')
Alternatively, we have a script sage-native-execute
sage-native-execute gedit
that is in SAGE_ROOT/local/bin/
that Sage itself uses whenever sage library code calls
system-wide binaries.
-- William
There is now a patch up at
http://trac.sagemath.org/sage_trac/ticket/3685
that fixes the above-mentioned problem.
William
This script just sets the LD_LIBRARY_PATH to "" (in my case). Indeed, the
sage: os.system('unset LD_LIBRARY_PATH; gedit')
works, but because the PATHs are set to use sage python, other
programs will not run anyway:
sage: os.system('unset LD_LIBRARY_PATH; mayavi2')
Traceback (most recent call last):
File "/usr/bin/mayavi2", line 3, in <module>
from enthought.mayavi.scripts import mayavi2
ImportError: No module named mayavi.scripts
256
Hm, Sage just needs LD_LIBRARY_PATH and PATH set to something, while
the rest of the system to something else.
Well, PATHs could be fixed somehow. The problem is LD_LIBRARY_PATH --
but I think there is something like chrpath, that could fix the
problem.
E.g. maybe it's possible to write a script that sets this in Sage
automatically on the first run. I just believe there must be a way to
get this fixed.
The other option is to use Sage debian packages.
> There is now a patch up at
> http://trac.sagemath.org/sage_trac/ticket/3685
> that fixes the above-mentioned problem.
Thanks for fixing this!
Ondrej
Sage should use its own versions of libraries for the reasons you
stated. One exception could be Python,
e.g. there could be an easy way to compile Sage with the systemwide
python, because (imho) Python doesn't change that much, so it could
work in most cases. It works even now.
As chatted on IRC, the LD_LIBRARY_PATH problem *might* get fixed using:
http://packages.debian.org/sid/chrpath
And the other PATH stuff are fixable too.
Ondrej
I don't understand the OSX problems about universal/non universal
python (ucs2 vs ucs4 surely can be fixed).
> Linux x86 and x86 it should mostly work, but on Linux Itanium it does
> not work since in the official 2.5.2 readline is busted. On Solaris
That really sucks. I hope upstream fixes that.
> and OSX there is also the multilib issue, i.e. 32 vs. 64 bit are
> supported on the same machine. So I disagree that it mostly works and
> I consider it a bad, bad idea to even give the user some simple option
> to make Sage build with the system python. It is trivial to do if you
> know your way around the Sage build system, but that actually means
> that one can likely fix issues oneself like in your case. Everybody
> else just has to play by the rules.
Well, if the situation is that the system wide python is so different
on different machines that it makes Sage fail, then it sucks. Imho.
>
> And I won't post the obvious one line diff to make Sage with system
> python at build time work. If you need to ask you shouldn't play with
> something that could cause endless trouble :p
I understand your point as the release manager, but from the users
point of view, if every program distributed all the libraries (python,
fortran, libxml, and all the other stuff that ships with sage), and it
would be incompatible with the other installations, as currently Sage
is incompatible with systemwide python as you stressed several times,
then it's bad, because I cannot mix Sage with other components on my
system. So it's Sage and the rest of the world.
>> As chatted on IRC, the LD_LIBRARY_PATH problem *might* get fixed using:
>>
>> http://packages.debian.org/sid/chrpath
>
> Yes, this certainly offers possibilities.
Fortunately.
Ondrej
We officially reported the bug to them via their tracker.
Then one Sage developer worked for a long time and
heroically found a fix and reported it too. I think it maybe
just takes a long time to filter through the system back
to the *deployed* RHEL and SLES linux boxes all over
the place. I.e., using the system-wide Python by
default just isn't an option. And of course a lot of users
want to install Sage without having to be root, and they
don't know anything about Python or installing it themselves.
Also let me add that the goal of Sage is to provide a viable alternative
to the Ma's and that all of those programs are self-contained. Our design
decisions are very much motivated by the Sage mission statement. We
have to be really focused on that goal, and consequently *not* worry about
certain other very worthy goals like nicely integrating as a library. That's
not to say that it isn't *wonderful* that some people like Tim Abbot are
doing an amazing job on the goal of integrating as a library. Speaking
of which, I think debian-sage is the perfect solution for you Ondrej.
Of course you know all about it since you're very involved in that effort.
(I get the sense it will be ready soon too.)
>> > And I won't post the obvious one line diff to make Sage with system
>> > python at build time work. If you need to ask you shouldn't play with
>> > something that could cause endless trouble :p
>>
>> I understand your point as the release manager, but from the users
>> point of view, if every program distributed all the libraries (python,
>> fortran, libxml, and all the other stuff that ships with sage), and it
>> would be incompatible with the other installations, as currently Sage
>> is incompatible with systemwide python as you stressed several times,
>> then it's bad, because I cannot mix Sage with other components on my
>> system. So it's Sage and the rest of the world.
>
> Well, Sage works because it is self contained and it is only possible
> to work because it is self contained. We can fix bugs rapidly because
> we nearly control 100% of the components and its dependencies of Sage.
> No one ever promised that Sage would play well with your system and
> installing Python packages into a Sage install is trivial. You can
> trivially use your system's Python, but I have no intentions of making
> it easy on you because other people will see the email and will shoot
> themselves in the foot. Even once Sage is in Debian proper I will not
> consider anything a bug unless it can be reproduced with a vanilla
> Sage build from sources. There is only one official Sage release and
> it will not be the one in any distribution. I seriously doubt that any
> distribution can keep up with the official Sage.
That said, it will be very good for there to be a Sage in Debian, and
I'm really looking forward to the day that happens. I know because
before for many many users I was personally a Linux user
for which the world of software was equal to the world of
packages available in rpm or deb's. I can understand that perspective,
and want to make using Sage easy for such people.
> One big issue I see: What are you going to do when the Sage project
> updates to Python 2.6 and then to 3.0 and Debian does not keep pace or
> the other way around when say Debian moved to Python 3 before we do.
> Then you just FUBARed your current Sage install and people will blame
> Sage and not their own stupidity. The same applies to the issue with
> symbols in libz that you initially reported. It is trivial to
> understand why it happens but again I suspect that many people will
> blame Sage and not even report it to the mailing list. And those are
> the people who might then go back to some other CAS and have the
> impression that Sage sucks. So in the end I see it like this: A minor
> inconvenience you can fix by thinking about it for a second vs. a
> potentially huge fuckup for loads of n00bs - I don't have to think
> about the choices here :)
I strongly agree with this. And it is people exactly not getting
this point that is I think a big problem that has really held back
open source math software from providing a genuine alternative
to the Ma's for power users like me or your typical calculus
student.
>> >> As chatted on IRC, the LD_LIBRARY_PATH problem *might* get fixed using:
>>
>> >>http://packages.debian.org/sid/chrpath
>>
>> > Yes, this certainly offers possibilities.
>>
>> Fortunately.
>
> Yes, I think we can fix that via some script to make you happy :)
>
>> Ondrej
>
> Cheers,
>
> Michael
> >
>
--
I see.
Unfortunately not lately, but I at least created the list and wiki
pages. So all credit goes to Tim. He has done a fenomenal job.
Yes, That should fix the problem. Nevertheless I still want to find
ways to easily test newest Sage and newest SymPy in some canonical way
without messing up with paths and shooting myself in the foot, to use
mabshoff's words.
users -> years? How many years were you using linux and why did you
switch to Mac OS X?
I am thinking of buying a macbook, but installing Debian on it. :)
> for which the world of software was equal to the world of
> packages available in rpm or deb's. I can understand that perspective,
> and want to make using Sage easy for such people.
I understand the other perspective too, and actually, I may endup
teaching some undergrad classes in a year or two, so I will use Sage
package, not Sage debian package, simply because the people will be
using windows, not Debian. Or they will use ubuntu, but still it's
easy to just download and untar. Even though unfortunately, on my
laptop it takes many minutes, maybe up to 15 min together
(download+untarring+first sage run).
>
>> One big issue I see: What are you going to do when the Sage project
>> updates to Python 2.6 and then to 3.0 and Debian does not keep pace or
>> the other way around when say Debian moved to Python 3 before we do.
>> Then you just FUBARed your current Sage install and people will blame
>> Sage and not their own stupidity. The same applies to the issue with
>> symbols in libz that you initially reported. It is trivial to
>> understand why it happens but again I suspect that many people will
>> blame Sage and not even report it to the mailing list. And those are
>> the people who might then go back to some other CAS and have the
>> impression that Sage sucks. So in the end I see it like this: A minor
>> inconvenience you can fix by thinking about it for a second vs. a
>> potentially huge fuckup for loads of n00bs - I don't have to think
>> about the choices here :)
>
> I strongly agree with this. And it is people exactly not getting
> this point that is I think a big problem that has really held back
> open source math software from providing a genuine alternative
> to the Ma's for power users like me or your typical calculus
> student.
I strongly agree with this too. Some people say I see things too much
in black and white (was it you?:) and they are right. I think that if
you download a program and untar it, it needs to just work without
setting anything. Maybe doing one "make", if there is no binary. If it
fails, it's a showstopper and I think it's a perfectly valid reason to
go to some other CAS that just works.
Nevertheless I also think that a good program first needs to satisfy
the above, but if it does and there are resources then it should be
also possible to tweak it in a way to do things that I want. If you
know what I mean. So I will continue crafting that little script that
will make me happy.
Ondrej
yes
> How many years were you using linux and why did you
> switch to Mac OS X?
I use Linux everyday, e.g., on sage.math, and have since 1991.
I just use Mac OS X for my laptop, since I find Linux on Laptops a bit painful.
I really meant more using Linux as more of a button pushing
end user than as somebody who would put a lot of work into building
software from source.
OK from me. Good luck avoiding the wrath of mabshoff :-)
-- William