Releasing Sage 5.9

[Jeroen Demeyer]

Sage 5.9 is now totally ready to be released.

Sources:
http://boxen.math.washington.edu/home/release/sage-5.9/

Binaries (all the usual binaries are built, except for the OS X 10.4 PPC
App version):
http://boxen.math.washington.edu/home/buildbot/binaries/sage/5.9/

[Minh Nguyen]

Hi Harald,

On Fri, May 3, 2013 at 4:20 PM, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
> Sage 5.9 is now totally ready to be released.

I've updated the repositories at

https://bitbucket.org/sagemath

I will no longer update the repositories at

http://hg.sagemath.org

The documentation is building and should be updated on the Sage
website in a few hours.

--
Regards,
Minh Van Nguyen
http://bit.ly/mvngu

[Harald Schilly]

On Fri, May 3, 2013 at 8:20 AM, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:

Sources:
http://boxen.math.washington.edu/home/release/sage-5.9/

Binaries (all the usual binaries are built, except for the OS X 10.4 PPC App version):
http://boxen.math.washington.edu/home/buildbot/binaries/sage/5.9/

website + mirror contain the new release. everything is indexed, compressed and removed to be ready to conquer the world ;)

h

[Nathann Cohen]

> website + mirror contain the new release. everything is indexed, compressed
> and removed to be ready to conquer the world ;)

Yeeeeeeeeeeeeeeeeeeeeehaaaawwwwwwwwwww !!!

Nathann

[John Cremona]

I am unable to run sage-5.9 after installing it in my /usr/local in
order to restart our server. I am doing the same as for previous sage
version for years.

I have lost track of all the different error messages I have had on
this but currently:

jec@fermat%pwd
/usr/local/sage/sage-5.9
jec@fermat%sudo ./sage -b
chmod: cannot access `./install': No such file or directory

----------------------------------------------------------
sage: Building and installing modified Sage library files.


Installing c_lib
scons: `install' is up to date.
/usr/local/sage/sage-5.9/local/bin/sage-build: line 36: ./install: No
such file or directory
Error installing modified sage library code.

I was only trying to do sage -b from the install directory because in
the same directory (and yes, I have channged all file ownershos
recursively to root:root and have made them all readable by all
users):


jec@fermat%./sage
----------------------------------------------------------------------
| Sage Version 5.9, Release Date: 2013-04-30 |
| Type "notebook()" for the browser-based notebook interface. |
| Type "help()" for help. |
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/sage/sage-5.9/local/bin/sage-ipython", line 6, in <module>
from sage.misc.interpreter import SageTerminalApp
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/interpreter.py",
line 67, in <module>
import os, log, re, new, sys
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/log.py",
line 66, in <module>
import misc
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/misc.py",
line 39, in <module>
import sage.misc.prandom as random
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/prandom.py",
line 58, in <module>
from sage.misc.randstate import current_randstate
ImportError: libcsage.so: cannot open shared object file: No such file
or directory
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/local/sage/sage-5.9/local/lib/python/atexit.py", line 24,
in _run_exitfuncs
func(*targs, **kargs)
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/temporary_file.py",
line 51, in delete_tmpfiles
from sage.misc.misc import SAGE_TMP
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/misc.py",
line 39, in <module>
import sage.misc.prandom as random
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/prandom.py",
line 58, in <module>
from sage.misc.randstate import current_randstate
ImportError: libcsage.so: cannot open shared object file: No such file
or directory
Error in sys.exitfunc:
Traceback (most recent call last):
File "/usr/local/sage/sage-5.9/local/lib/python/atexit.py", line 24,
in _run_exitfuncs
func(*targs, **kargs)
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/temporary_file.py",
line 51, in delete_tmpfiles
from sage.misc.misc import SAGE_TMP
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/misc.py",
line 39, in <module>
import sage.misc.prandom as random
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/prandom.py",
line 58, in <module>
from sage.misc.randstate import current_randstate
ImportError: libcsage.so: cannot open shared object file: No such file
or directory

This is with a successful build from source which passed all make
ptestlong tests (all built and tested in my home directory as usual),
then I installed my usual set of spkgs.

John

> --
> You received this message because you are subscribed to the Google Groups "sage-release" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-release...@googlegroups.com.
> To post to this group, send email to sage-r...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-release?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[William Stein]

And what is that set, so cell it obviously caused the problem.    I had the same experience with 5.9 + my optional packages = broken.

[John Cremona]

On 3 May 2013 20:40, William Stein <wst...@gmail.com> wrote:
>
> On May 3, 2013 12:15 PM, "John Cremona" <john.c...@gmail.com> wrote:
>>
>> I am unable to run sage-5.9 after installing it in my /usr/local in
>> order to restart our server. I am doing the same as for previous sage
>> version for years.
>>

>>

>> This is with a successful build from source which passed all make
>> ptestlong tests (all built and tested in my home directory as usual),
>> then I installed my usual set of spkgs.
>>
>
> And what is that set, so cell it obviously caused the problem. I had the
> same experience with 5.9 + my optional packages = broken.

database_cremona_ellcurve-20121022.spkg gap_packages-4.5.7.spkg
pyopenssl-0.13.p0.spkg
database_gap-4.5.7.spkg lie-2.2.2.p5.spkg

and also I had installed these:

easy_install -U flask
easy_install -U flask-login
easy_install -U pymongo
easy_install -U flask-markdown
easy_install -U flask-cache
easy_install -U pyyaml


>
>> John
>>

[William Stein]

What happens when you type

./sage -br

> --
> You received this message because you are subscribed to the Google Groups "sage-release" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-release...@googlegroups.com.
> To post to this group, send email to sage-r...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-release?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--

William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[leif]

John Cremona wrote:
> I am unable to run sage-5.9 after installing it in my /usr/local in
> order to restart our server. I am doing the same as for previous sage
> version for years.

I of course don't know what you did for years... ;-)

Did you build Sage somewhere and afterwards *move* the installation, or
did you somehow *copy* it?


Apparently some file(s) in $SAGE_ROOT/devel/sage/ are missing, namely at
least the 'install' script. (Note that 'sage' there should be a
[relative] symbolic link to the directory 'sage-main' in the same
folder, $SAGE_ROOT/devel/.)


-leif

--
() The ASCII Ribbon Campaign
/\ Help Cure HTML E-Mail

[Harald Schilly]

hg.sagemath.org now properly points to bitbucket.org too ;)

h

[John Cremona]

On 3 May 2013 21:10, William Stein <wst...@gmail.com> wrote:
> What happens when you type
>
> ./sage -br

jec@fermat%./sage -br

chmod: cannot access `./install': No such file or directory

----------------------------------------------------------
sage: Building and installing modified Sage library files.


Installing c_lib

scons: *** [src/convert.c] .sconsign.dblite: Permission denied
Error building c_lib.

[John Cremona]

On 3 May 2013 21:19, leif <not.r...@online.de> wrote:
> John Cremona wrote:
>>
>> I am unable to run sage-5.9 after installing it in my /usr/local in
>> order to restart our server. I am doing the same as for previous sage
>> version for years.
>
>
> I of course don't know what you did for years... ;-)

.. but it used to work.

>
> Did you build Sage somewhere and afterwards *move* the installation, or did
> you somehow *copy* it?

I used cp -a to copy the whole directory after building and installing
optional spkgs.

>
>
> Apparently some file(s) in $SAGE_ROOT/devel/sage/ are missing, namely at
> least the 'install' script. (Note that 'sage' there should be a [relative]
> symbolic link to the directory 'sage-main' in the same folder,
> $SAGE_ROOT/devel/.)
>
>

I deleted that copy and recopied to make sure I had not messed that up. Now:

running /usr/local/sage/sage-5.9/sage runs fine, but

running my script to start the server:

/usr/local/bin/sage -notebook interface='' accounts=True secure=True
automatic_login=False

(where /usr/local/bin/sage is a link to /usr/local/sage/sage-5.9/sage)
gives errors:

Please wait while the Sage Notebook server starts...

Traceback (most recent call last):

File "/usr/local/sage/sage-5.9/local/bin/sage-notebook", line 9, in <module>
from sage.all import *
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/all.py",
line 82, in <module>
from sage.misc.all import * # takes a while
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/all.py",
line 1, in <module>
from misc import (alarm, ellipsis_range, ellipsis_iter, srange,
xsrange, sxrange, getitem,
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/misc.py",
line 2082, in <module>
import sage.server.support
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/server/support.py",
line 19, in <module>
import sage.misc.sagedoc as sagedoc
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/sagedoc.py",
line 40, in <module>
from sagenb.misc.sphinxify import sphinxify
ImportError: No module named sagenb.misc.sphinxify

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/local/sage/sage-5.9/local/lib/python/atexit.py", line 24,
in _run_exitfuncs
func(*targs, **kargs)
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/temporary_file.py",
line 51, in delete_tmpfiles
from sage.misc.misc import SAGE_TMP
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/misc.py",

line 2082, in <module>
import sage.server.support
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/server/support.py",
line 19, in <module>
import sage.misc.sagedoc as sagedoc
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/sagedoc.py",
line 40, in <module>
from sagenb.misc.sphinxify import sphinxify
ImportError: No module named sagenb.misc.sphinxify

Error in sys.exitfunc:
Traceback (most recent call last):
File "/usr/local/sage/sage-5.9/local/lib/python/atexit.py", line 24,
in _run_exitfuncs
func(*targs, **kargs)
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/temporary_file.py",
line 51, in delete_tmpfiles
from sage.misc.misc import SAGE_TMP
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/misc.py",

line 2082, in <module>
import sage.server.support
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/server/support.py",
line 19, in <module>
import sage.misc.sagedoc as sagedoc
File "/usr/local/sage/sage-5.9/local/lib/python2.7/site-packages/sage/misc/sagedoc.py",
line 40, in <module>
from sagenb.misc.sphinxify import sphinxify
ImportError: No module named sagenb.misc.sphinxify

[leif]

Hmmm, then something went wrong copying $SAGE_ROOT/devel/sagenb (or the
directory it points to, 'sagenb-main'), too.

Did the 'cp' actually succeed without any errors?

Assuming you get the same import error when running

$ /usr/local/sage/sage-5.9/sage -c 'from sagenb.misc.sphinxify import
sphinxify'

...

Really smells like some filesystem issue (disk full?).


-leif

[Paul-Olivier Dehaye]

Sometimes when you copy sage after installation the cp seemingly works but things break because the permissions are subtly changed. For me it was due to an issue with the shared file server (copying from one drive to another). 

Paul

--
You received this message because you are subscribed to the Google Groups "sage-release" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-release+unsubscribe@googlegroups.com.

[Jeroen Demeyer]

On 05/03/2013 11:54 PM, leif wrote:
> Did the 'cp' actually succeed without any errors?
>

> Really smells like some filesystem issue (disk full?).

Good bet, my guess would also go in this direction.

[John Cremona]

On 4 May 2013 07:10, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
> On 05/03/2013 11:54 PM, leif wrote:
>>
>> Did the 'cp' actually succeed without any errors?

Yes (but see below).

>>
>> Really smells like some filesystem issue (disk full?).
>
> Good bet, my guess would also go in this direction.
>

No way, there are 200gb free in /usr/local/.

I just did a recursive diff in the original build dir against the
copied dir, and after ignoring differing binary files (pyc mostly) and
different paths (which get fixed the first time Sage is run in the new
place) there are a whole lot of files which were not copied, all in
/home/jec/sage-5.9/local/lib/python and python2.7/site-packages,
relating to the stuff I easy-installed.

Clue: using my own login this works:
jec@fermat%/usr/local/sage/sage-5.9/sage

----------------------------------------------------------------------
| Sage Version 5.9, Release Date: 2013-04-30 |
| Type "notebook()" for the browser-based notebook interface. |
| Type "help()" for help. |
----------------------------------------------------------------------

sage:

but using my sage user (which runs the server):


sage@fermat:~$ /usr/local/sage/sage-5.9/sage

----------------------------------------------------------------------
| Sage Version 5.9, Release Date: 2013-04-30 |
| Type "notebook()" for the browser-based notebook interface. |
| Type "help()" for help. |
----------------------------------------------------------------------

Traceback (most recent call last):

...

ImportError: No module named sagenb.misc.sphinxify

Now I am sure that you will say "Wrong file permissions" but I have
checked that all are ok (I use sudo chmod -R a+rX
/usr/local/sage/sage-5.9).

Perhaps there is some difference in the ~/.sage -- I have deleted all
in ~/.sage/temp and tmp but that dir not help.

Thanks for the suggestions so far.

John

[leif]

John Cremona wrote:
> On 4 May 2013 07:10, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>> On 05/03/2013 11:54 PM, leif wrote:
>>>
>>> Did the 'cp' actually succeed without any errors?
>
> Yes (but see below).
>
>>>
>>> Really smells like some filesystem issue (disk full?).
>>
>> Good bet, my guess would also go in this direction.
>>
>
> No way, there are 200gb free in /usr/local/.

Sage's quota exceeded? ;-)

Then something may have messed up your
$SAGE_ROOT/local/lib/python/site-packages/easy-install.pth; is the
sagenb path there still relative (assuming the file got correctly copied
to /usr/local/...)?

The sagenb entry should read

../../../../devel/sagenb


Even that doesn't explain why $SAGE_ROOT/devel/sage/install (and
probably more) was missing in the copy in the first place.


Are all files below /usr/local/sage/sage-5.9/ owned by the user 'sage'?


Does the 'sage' user set any suspicious environment variables in his/her
.login / .profile / .*rc files? (Although you said you didn't get this
trouble with previous releases, some of /these/ files may have changed
inbetween.)


-leif

> Perhaps there is some difference in the ~/.sage -- I have deleted all
> in ~/.sage/temp and tmp but that dir not help.
>
> Thanks for the suggestions so far.
>
> John

--

[Jeroen Demeyer]

John, perhaps a good test is to reinstall Sage 5.8 now and check whether
that reinstall still works. If that works and a reinstall of Sage 5.9
doesn't, then we know that it's really a Sage-related problem.

[John Cremona]

The 5.8 installation is still there and still works and in fact I
restarted it so that our users would not complain.

I'll look into Leif's other suggestions later, but none of them seem
likely as we don't have quotas and I only ever log in as the sage user
when I have just installed a new version and want to restart the
server.

The /usr/local/sage/sage-5.9 files all belong to root but are
world-readable. And they work from my own username (which did the
build) but not from the sage username.

Jeroen has an account on this machine (fermat.warwick.ac.uk) and can
try it out if he wants to!

John

[Justin C. Walker]

On May 4, 2013, at 14:42 , John Cremona <john.c...@gmail.com> wrote:

> The 5.8 installation is still there and still works and in fact I
> restarted it so that our users would not complain.
>
> I'll look into Leif's other suggestions later, but none of them seem
> likely as we don't have quotas and I only ever log in as the sage user
> when I have just installed a new version and want to restart the
> server.

FWIW, 'python' is usually (in the absence of user intervention) a symbolic link to 'python-version-in-use', 2.7 in this case. The fact that files are missing there is probably the root of the problem (unless it's me that's missing something). As I understand this, Python looks in 'lib/python/' for its files, not 'python-version'. Sphinxify is located in (looked for in) some file in 'lib/python/site-packages'.

Justin

--
Justin C. Walker, Curmudgeon at Large
Institute for the Absorption of Federal Funds
-----------
I want to die, peacefully in my sleep, like my grandfather;
not screaming in terror, like his passengers.

[John Cremona]

On 4 May 2013 23:34, Justin C. Walker <jus...@mac.com> wrote:
>
> On May 4, 2013, at 14:42 , John Cremona <john.c...@gmail.com> wrote:
>
>> The 5.8 installation is still there and still works and in fact I
>> restarted it so that our users would not complain.
>>
>> I'll look into Leif's other suggestions later, but none of them seem
>> likely as we don't have quotas and I only ever log in as the sage user
>> when I have just installed a new version and want to restart the
>> server.
>
> FWIW, 'python' is usually (in the absence of user intervention) a symbolic link to 'python-version-in-use', 2.7 in this case. The fact that files are missing there is probably the root of the problem (unless it's me that's missing something). As I understand this, Python looks in 'lib/python/' for its files, not 'python-version'. Sphinxify is located in (looked for in) some file in 'lib/python/site-packages'.
>

SUCCESS! See below.

The copy in /usr/local I am having issues with has local/lib/python
correctly linked to python2.7 . And it works fine when run as user
jec (me) just not when run as user sage.

Here's another clue. If I run /usr/local/sage/sage-5.9/sage as myself
it just runs. If I do

sudo /usr/local/sage/sage-5.9/sage -b

then it succeeds (doing nothing much as nothing needs rebuilding) but
the output includes the lines

warning: Replacing library search directory in linker command:
"/home/jec/sage-5.9/local/lib" -> "/usr/local/sage/sage-5.9/local/lib"

which suggests to be that the code which is run after relocating
(copying to /usr/local) is imperfect. And YES: I go back to the
original copy in /home/jec/sage5.9 and make all files world-readable
recursively, and NOW the sage user can successfully run
/usr/local/sage/sage-5.9/sage.

I can see that one solution for the future would be to always build
sage in the same place
where it will be run, or at least in the home directory of the sage
user whcih will be running the notebook server, but surely we *must*
support relocation (as in "make install" for most software) after
building, and this worked up to 5.8 but is apparently broken now.

John

> Justin
>
> --
> Justin C. Walker, Curmudgeon at Large
> Institute for the Absorption of Federal Funds
> -----------
> I want to die, peacefully in my sleep, like my grandfather;
> not screaming in terror, like his passengers.
>
>
>
>

[Volker Braun]

On Sunday, May 5, 2013 12:18:14 PM UTC+1, John Cremona wrote:

but surely we *must*
support relocation (as in "make install" for most software) after
building,

This is not relocation, you need to give the destination directory to "configure --prefix=..." and "make install" then installs into the prefix. In particular, libtools will relink shared libraries in the prefix directory if necessary.

The relocation stuff in Sage is somewhere between naive and dangerous. In particular, if you build Sage in /tmp and then move it elsewhere its trivial to inject code into other's Sage session through dangling rpaths.

[John Cremona]

OK, so I will change my normal procedure in installing new versions of
sage globally on any multi-user system: I will unpack and build
directly into /usr/local/sage and do no copying or moving.

John

[leif]

John Cremona wrote:
> On 4 May 2013 23:34, Justin C. Walker <jus...@mac.com> wrote:
>>
>> On May 4, 2013, at 14:42 , John Cremona <john.c...@gmail.com> wrote:
>>
>>> The 5.8 installation is still there and still works and in fact I
>>> restarted it so that our users would not complain.
>>>
>>> I'll look into Leif's other suggestions later, but none of them seem
>>> likely as we don't have quotas and I only ever log in as the sage user
>>> when I have just installed a new version and want to restart the
>>> server.
>>
>> FWIW, 'python' is usually (in the absence of user intervention) a symbolic link to 'python-version-in-use', 2.7 in this case. The fact that files are missing there is probably the root of the problem (unless it's me that's missing something). As I understand this, Python looks in 'lib/python/' for its files, not 'python-version'. Sphinxify is located in (looked for in) some file in 'lib/python/site-packages'.
>>
>
> SUCCESS! See below.
>
> The copy in /usr/local I am having issues with has local/lib/python
> correctly linked to python2.7 . And it works fine when run as user
> jec (me) just not when run as user sage.
>
> Here's another clue. If I run /usr/local/sage/sage-5.9/sage as myself
> it just runs. If I do
>
> sudo /usr/local/sage/sage-5.9/sage -b
>
> then it succeeds (doing nothing much as nothing needs rebuilding) but
> the output includes the lines
>
> warning: Replacing library search directory in linker command:
> "/home/jec/sage-5.9/local/lib" -> "/usr/local/sage/sage-5.9/local/lib"

That's pretty harmless, or more precisely, we added this warning when we
fixed that relocation issue... (which at that time only appeared on
MacOS X IIRC).

But maybe you're right and we'd have to do more in conjunction with some
optional spkgs...

> which suggests to be that the code which is run after relocating
> (copying to /usr/local) is imperfect. And YES: I go back to the
> original copy in /home/jec/sage5.9 and make all files world-readable
> recursively, and NOW the sage user can successfully run
> /usr/local/sage/sage-5.9/sage.

Oh, I forgot to suggest to temporarily rename the folder of your
original installation (in /home/jec), such that the dynamic
linker/loader would definitely complain if any file (executable, shared
library) in /usr/local/sage would still refer to shared libraries in the
original (now renamed) build tree.

> I can see that one solution for the future would be to always build
> sage in the same place
> where it will be run, or at least in the home directory of the sage
> user whcih will be running the notebook server, but surely we *must*
> support relocation (as in "make install" for most software) after
> building, and this worked up to 5.8 but is apparently broken now.

Hmmm, yes, "relocation" of Sage is supposed to at least work (modulo the
security issues Volker mentioned -- which can be avoided by taking some
care -- as some RPATHs might still contain the original build location,
which btw. is relevant especially when preparing bdists as well).


It's still not immediately clear to me what actually triggered your
problem, as you apparently ran into a couple of issues and hence tried a
lot on the way. So someone^TM should try to reproduce your problem.

Can you give a list of the optional spkgs you install, or the steps you
used to perform in the past?


-leif

[John Cremona]

>
> Hmmm, yes, "relocation" of Sage is supposed to at least work (modulo the
> security issues Volker mentioned -- which can be avoided by taking some care
> -- as some RPATHs might still contain the original build location, which
> btw. is relevant especially when preparing bdists as well).
>
>
> It's still not immediately clear to me what actually triggered your problem,
> as you apparently ran into a couple of issues and hence tried a lot on the
> way. So someone^TM should try to reproduce your problem.
>
> Can you give a list of the optional spkgs you install, or the steps you used
> to perform in the past?
>

I am probably not consistent but these are the ones installed now:

database_cremona_ellcurve-20121022.spkg
gap_packages-4.5.7.spkg
pyopenssl-0.13.p0.spkg
database_gap-4.5.7.spkg
lie-2.2.2.p5.spkg

and I had the same in my 5.8 global installation. It is also quite
likely that I have not always python-installed all the same modules
(flask, pymongo etc) which I use for the lmfdb project.

By now I have rebuilt from scratch in the place where I wanted the
global installation and that seems to be fine. I'll do that in
future.

John

>
> -leif
>
>
> --
> () The ASCII Ribbon Campaign
> /\ Help Cure HTML E-Mail
>

[William Stein]

Hi,

I've upgrade https://cloud.sagemath.com to sage-5.9 with the following
optional packages, in case anybody wants to just try out something in
sage-5.9 (or an optional spkg). Are there any other optional packages
I should install? Also, if you try out https://cloud.sagemath.com, is there
anything else system-wide that I should always make available (e.g., I
installed Macaulay2, Octave, Scilab, Axiom, etc.).

4ti2-1.3.2.p1
biopython-1.60
brian-1.2.1.p0
cbc-2.7.5
cluster_seed-1.0
coxeter3-1.1
cryptominisat-2.9.6
cunningham_tables-1.0
database_cremona_ellcurve-20121022
database_gap-4.5.7
database_jones_numfield-v4
database_kohel-20060803
database_odlyzko_zeta
database_sloane_oeis-2005-12
database_symbolic_data-20070206
dot2tex-2.8.7-2
gap_packages-4.5.7
gnuplotpy-1.8
guppy-0.1.8
kash3-2008-07-31.p0
lie-2.2.2.p5
lrs-4.2b.p1
nauty-24
normaliz-2.8.p0
nose-1.1.2
nzmath-1.1.0
p_group_cohomology-2.1.3
phc-2.3.65.p0
pybtex-20120618
pycryptoplus-20100809-git
pyx-0.10
pyzmq-2.1.11.p1
qhull-2010.1
sage-mode-0.7
TOPCOM-0.17.4
zeromq-2.2.0.p0

[Simon King]

Hi William,

On 2013-05-06, William Stein <wst...@gmail.com> wrote:
> p_group_cohomology-2.1.3

Thank you! But version 2.1.3 of my package won't work with Sage-5.8 or
later, because of some internal changes made in Sage. Version 2.1.4 of
my package copes with these changes and almost has a positive review,
see #14492.

Cheers,
Simon

[Harald Schilly]

On Monday, May 6, 2013 4:53:11 PM UTC+2, William Stein wrote:

Also, if you try out https://cloud.sagemath.com, is there

anything else system-wide that I should always make available?

Last time I tried it I wanted to install some python packages. I.e.  the ones for data analysis and statistics: pandas, statsmodels, pytables, etc.

H

[William Stein]

Harald,

I attempted to install pandas into sage-5.9 on Ubuntu 12.04 LTS, and
it fails immediately with tons of errors, starting with this (see
below). Have you ever actually installed pandas into Sage? Has
anybody?

building 'pandas.index' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/pandas
creating build/temp.linux-x86_64-2.7/pandas/src
creating build/temp.linux-x86_64-2.7/pandas/src/datetime
gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -fPIC -Ipandas/src/klib -Ipandas/src
-I/usr/local/sage/sage-5.9/
local/lib/python2.7/site-packages/numpy/core/include
-I/usr/local/sage/sage-5.9/local/include/python2.7 -c pandas/index.c
-o build/temp.linux-x86_
64-2.7/pandas/index.o
In file included from pandas/index.c:260:0:
pandas/src/numpy_helper.h: In function ‘infer_type’:
pandas/src/numpy_helper.h:32:12: error: ‘PyDatetimeArrType_Type’
undeclared (first use in this function)
pandas/src/numpy_helper.h:32:12: note: each undeclared identifier is
reported only once for each function it appears in
pandas/src/numpy_helper.h: In function ‘get_datetime64_value’:
pandas/src/numpy_helper.h:53:12: error: ‘PyDatetimeScalarObject’
undeclared (first use in this function)
pandas/src/numpy_helper.h:53:35: error: expected expression before ‘)’
token
pandas/src/numpy_helper.h: In function ‘is_datetime64_object’:
pandas/src/numpy_helper.h:84:10: error: ‘PyDatetimeArrType_Type’
undeclared (first use in this function)
pandas/src/numpy_helper.h: In function ‘is_timedelta64_object’:
pandas/src/numpy_helper.h:89:10: error: ‘PyTimedeltaArrType_Type’
undeclared (first use in this function)
In file included from pandas/src/klib/khash_python.h:3:0,
from pandas/index.c:261:


THEN, I tried the next older version of pandas (0.10.1), and it fails with:

(sage-sh) salvus@salvus-base:pandas-0.10.1$ python setup.py build
pandas requires NumPy >= 1.6.1 due to datetime64 dependency
(sage-sh) salvus@salvus-base:pandas-0.10.1$ sage -standard |grep numpy
numpy-1.5.1.p1

Evidently, the numpy in Sage-5.9 is old. Also, evidently the pandas
people forgot to check the numpy dep in version 0.11, but had it in
0.10.1. I get the same dep error all the way back to 0.8 (from a
year ago), but 0.7.3 works, so now pandas-0.7.3 is installed on
https://cloud.sagemath.com/.

Anyway, we clearly need to upgrade numpy!?



William

> H

[William Stein]

On Tue, May 14, 2013 at 8:01 AM, William Stein <wst...@gmail.com> wrote:
> On Wed, May 8, 2013 at 12:43 PM, Harald Schilly
> <harald....@gmail.com> wrote:
>>
>>
>> On Monday, May 6, 2013 4:53:11 PM UTC+2, William Stein wrote:
>>>
>>> Also, if you try out https://cloud.sagemath.com, is there
>>> anything else system-wide that I should always make available?
>>
>>
>> Last time I tried it I wanted to install some python packages. I.e. the
>> ones for data analysis and statistics: pandas, statsmodels, pytables, etc.
>>
>

Regarding pytables, I had to go back to version 2.2.1 from 2010 (!) in
order to get something that would build with Sage.

Conclusion: Sage is useless for modern Python-based stats work.

Um, we should fix this!

William

[Harald Schilly]

On Tue, May 14, 2013 at 5:01 PM, William Stein <wst...@gmail.com> wrote:
> Have you ever actually installed pandas into Sage?

No, and actually I should have tried this first before asking about
this … thank's for looking into this, and unless numpy is updated to
get this working it looks like a showstopper to me :\

Harald

[William Stein]

On Tue, May 14, 2013 at 8:09 AM, William Stein <wst...@gmail.com> wrote:
> On Tue, May 14, 2013 at 8:01 AM, William Stein <wst...@gmail.com> wrote:
>> On Wed, May 8, 2013 at 12:43 PM, Harald Schilly
>> <harald....@gmail.com> wrote:
>>>
>>>
>>> On Monday, May 6, 2013 4:53:11 PM UTC+2, William Stein wrote:
>>>>
>>>> Also, if you try out https://cloud.sagemath.com, is there
>>>> anything else system-wide that I should always make available?
>>>
>>>
>>> Last time I tried it I wanted to install some python packages. I.e. the
>>> ones for data analysis and statistics: pandas, statsmodels, pytables, etc.
>>>
>>
>
> Regarding pytables, I had to go back to version 2.2.1 from 2010 (!) in
> order to get something that would build with Sage.

And it turns out that though it *builds* with Sage, it simply moves
the error to runtime:

sage: import tables
*Fatal error*: You need Numexpr >= 1.4.1 for running PyTables!
An exception has occurred, use %tb to see the full traceback.

---

Anyway, I guess it is interesting to me that maybe nobody has ever
used Sage + Python stats libraries together, ever.

William

>
> Conclusion: Sage is useless for modern Python-based stats work.
>
> Um, we should fix this!
>
> William

[leif]

Harald Schilly wrote:
> On Tue, May 14, 2013 at 5:01 PM, William Stein <wst...@gmail.com> wrote:
>> Have you ever actually installed pandas into Sage?
>
> No, and actually I should have tried this first before asking about

> this � thank's for looking into this, and unless numpy is updated to

> get this working it looks like a showstopper to me :\

NumPy 1.7.0 has been merged into Sage 5.10.beta0 [1], just saying...


-leif

[1] http://trac.sagemath.org/sage_trac/ticket/11334

[William Stein]

On Tue, May 14, 2013 at 8:41 AM, leif <not.r...@online.de> wrote:
> Harald Schilly wrote:
>>
>> On Tue, May 14, 2013 at 5:01 PM, William Stein <wst...@gmail.com> wrote:
>>>
>>> Have you ever actually installed pandas into Sage?
>>
>>
>> No, and actually I should have tried this first before asking about

>> this … thank's for looking into this, and unless numpy is updated to

>> get this working it looks like a showstopper to me :\
>
>
> NumPy 1.7.0 has been merged into Sage 5.10.beta0 [1], just saying...

AWESOME! Thanks to François Bissey!

>
>
> -leif
>
> [1] http://trac.sagemath.org/sage_trac/ticket/11334
>
>
> --
> () The ASCII Ribbon Campaign
> /\ Help Cure HTML E-Mail
>

[William Stein]

On Tue, May 14, 2013 at 8:46 AM, William Stein <wst...@gmail.com> wrote:
> On Tue, May 14, 2013 at 8:41 AM, leif <not.r...@online.de> wrote:
>> Harald Schilly wrote:
>>>
>>> On Tue, May 14, 2013 at 5:01 PM, William Stein <wst...@gmail.com> wrote:
>>>>
>>>> Have you ever actually installed pandas into Sage?
>>>
>>>
>>> No, and actually I should have tried this first before asking about
>>> this … thank's for looking into this, and unless numpy is updated to
>>> get this working it looks like a showstopper to me :\
>>
>>
>> NumPy 1.7.0 has been merged into Sage 5.10.beta0 [1], just saying...
>
> AWESOME! Thanks to François Bissey!
>

I've upgraded https://cloud.sagemath.com to use this numpy 1.7.0 (and
sage-5.10.beta3), and now pytables, pandas, and statsmodels all
install no problem.... so long as you use pip (not easy_install) to
install them.

We should really get rid of easy_install, include pip in Sage, and
make it so "sage -i package_name" can pull modules from pypi, if they
aren't sage-hosted....

William

[Harald Schilly]

On Fri, May 17, 2013 at 3:29 AM, William Stein <wst...@gmail.com> wrote:
> I've upgraded https://cloud.sagemath.com to use this numpy 1.7.0 (and
> sage-5.10.beta3), and now pytables, pandas, and statsmodels all
> install no problem.... so long as you use pip (not easy_install) to
> install them.

Thanks! I just had time to look into this.

Below I'm running their unit tests. Besides checking if they (mostly)
do something useful, that's probably also a nice load test for the
server infrastructure:

First, I started a shell and inside it, I started ipython. Then:

import sklearn
sklearn.test()
gives about 15 of ~1700 test failures.

numpy 1.7: zero failures!

import statsmodels
statsmodels.test()
At least 32 out of about 300 fail and the module "patsy" is missing.
This is a dependency that allows to formulate "R-style" statistical
formulas in python.
http://patsy.readthedocs.org/en/latest/
so, the interesting question is, should this dependency be installed
automatically … ?

pandas is tested this way from inside ipython:
import pandas
p = pandas.__path__[0]
!nosetests $p
gives: "OK" … 3221 tests with 0 failures.

import tables
tables.test()
# only the "light" test, the heavy one is supposed to take many hours
12 errors (all the same mistake) out of about 5000 tests.

So, this looks really good!

Harald

[William Stein]

On Wed, May 29, 2013 at 5:56 AM, Harald Schilly
<harald....@gmail.com> wrote:
> On Fri, May 17, 2013 at 3:29 AM, William Stein <wst...@gmail.com> wrote:
>> I've upgraded https://cloud.sagemath.com to use this numpy 1.7.0 (and
>> sage-5.10.beta3), and now pytables, pandas, and statsmodels all
>> install no problem.... so long as you use pip (not easy_install) to
>> install them.
>
>
> Thanks! I just had time to look into this.
>
> Below I'm running their unit tests. Besides checking if they (mostly)
> do something useful, that's probably also a nice load test for the
> server infrastructure:

How did that go?

>
> First, I started a shell and inside it, I started ipython. Then:
>
> import sklearn
> sklearn.test()
> gives about 15 of ~1700 test failures.

Is this bad/normal/good for sklearn? I know if it were Sage, having
15 failures could be pretty serious.

>
> numpy 1.7: zero failures!
>
> import statsmodels
> statsmodels.test()
> At least 32 out of about 300 fail and the module "patsy" is missing.
> This is a dependency that allows to formulate "R-style" statistical
> formulas in python.
> http://patsy.readthedocs.org/en/latest/
> so, the interesting question is, should this dependency be installed
> automatically … ?

Yes, definitely. Thanks for suggesting it. It'll be there when I
next update sage on the vm's. It's very easy to install with pip.

> pandas is tested this way from inside ipython:
> import pandas
> p = pandas.__path__[0]
> !nosetests $p
> gives: "OK" … 3221 tests with 0 failures.
>
> import tables
> tables.test()
> # only the "light" test, the heavy one is supposed to take many hours
> 12 errors (all the same mistake) out of about 5000 tests.
>
> So, this looks really good!

It could certainly be worse, though the existence of failing tests
might worry me (?).

I wonder how https://www.wakari.io/ environment does with the same tests...

Thanks! I've been planning to have a script that runs a long
sequence of tests of as many things as possible in the "super sage"
distribution in Salvus, and the above will help a lot.

-- William

>
> Harald

[Harald Schilly]

On Mon, Jun 3, 2013 at 5:57 AM, William Stein <wst...@gmail.com> wrote:
>> gives about 15 of ~1700 test failures.
>
> Is this bad/normal/good for sklearn? I know if it were Sage, having
> 15 failures could be pretty serious.

well, i have no idea, i also don't remember what they were. but more
general, they exercise test driven development and they (and most of
the others packages) use travis-ci.
their config file:
https://github.com/scikit-learn/scikit-learn/blob/master/.travis.yml

and latest status messages suggst, that usually everything passes:
https://travis-ci.org/scikit-learn/scikit-learn/builds

> Thanks!

glad to be of help :-)

h