recent woes with relocation of SageMath installation directory

[Samuel Lelièvre]

Dear sage-support (cc: sage-release, sage-devel)
(please reply on sage-support only)

I received the following question after announcing
Sage 7.0 on sage-announce.

This is one of many such questions I am reading on
various lists and sites recently.

Something needs to be more clearly documented,
or refactored.

Can someone answer this question by replying to
this post on sage-support?

Thanks!
Samuel

---------- Forwarded message ----------
Date: Mon, 1 Feb 2016 23:32:31 -0600
To: Samuel Lelievre <samuel....@gmail.com>
Subject: Installing upgraded SageMath

I tried installing sagemath-upstream-binary on my Ubuntu Trusty Tahr
system and saw this:

Rewriting paths for your new installation directory
===================================================

This might take a few minutes but only has to be done once.

patching /usr/lib/sagemath/src/build/cythonized/sage/structure/list_clone.c
Traceback (most recent call last):
File "/usr/lib/sagemath/relocate-once.py", line 85, in <module>
p('src/build/cythonized/sage/structure/list_clone.c').substitute().save()
File "/usr/lib/sagemath/relocate-once.py", line 70, in __call__
return FilePatch(self, filename)
File "/usr/lib/sagemath/relocate-once.py", line 21, in __init__
self.st = os.stat(filename)
OSError: [Errno 2] No such file or directory:
'/usr/lib/sagemath/src/build/cythonized/sage/structure/list_clone.c'

Any idea what is up?

Dale Amon

[Volker Braun]

Is this the Ubuntu PPA? I think thats not working right now. Whoever is maintaining it (Jan?) should probably revert it until its fixed.

[Jan Groenewald]

Hi

Yes, that is the Ubuntu PPA. The installation is mostly working despite that error message.

(We have it running on campus while courses are using sage for the last 3 weeks.)

I'm waiting to see a use case that warrants reverting, despite the lack of confidence that this message inspires.

Regards,

Jan

--
You received this message because you are subscribed to the Google Groups "sage-release" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-release...@googlegroups.com.
To post to this group, send email to sage-r...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-release.
For more options, visit https://groups.google.com/d/optout.

--

  .~.
  /V\     Jan Groenewald
 /( )\    www.aims.ac.za
 ^^-^^ 

[Henri Girard]

I installed binary from sagemath web site 64 ubuntu 15.10 on xenial it works fine. I couldn't get ppa aims working.

--
You received this message because you are subscribed to the Google Groups "sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-support...@googlegroups.com.
To post to this group, send email to sage-s...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-support.

[Volker Braun]

If its working (i.e. the paths are already correct) then just delete the relocate-once.py script.It would delete itself anyways after successful execution.

[Jan Groenewald]

Hi

TL;DR install sagemath-upstream-binary-full for a fix. The error in 6.9 is harmless as only one unimportant file is missing.

The development PPA now has a working sagemath-upstream-binary-full 7.0 with no error at all.

https://launchpad.net/~aims/+archive/ubuntu/sagemath-dev
When it runs the relocate-once.py after install, it takes (quite) a while to patch and then prints message:

...
patching /usr/lib/sagemath/local/lib/python2.7/site-packages/sage/ext/pselect.so
patching /usr/lib/sagemath/local/lib/libstdc++.so.6.0.20
* AIMS PPA: relocate-once.py finished running. Ignore above Exception
┌────────────────────────────────────────────────────────────────────┐
│ SageMath Version 7.0, Release Date: 2016-01-19                     │
│ Type "notebook()" for the browser-based notebook interface.        │
│ Type "help()" for help.                                            │
└────────────────────────────────────────────────────────────────────┘
sage: Exiting Sage (CPU time 0m0.02s, Wall time 0m0.10s).
Setting up sagemath-upstream-binary-full (7.0~aimsppa1~qa201602031255151) ...
0 root@muizenberg:~#

Even though there is no exception; I added that warning before putting this out. This is because I added one missing file to sagemath-binary-upstream ($SAGE_ROOT/builId/make/Makefile). This version will be uploaded to the stable PPA in a week or two depending on feedback.

The smaller bandwidth-saving and disk-saving sagemath-upstream-binary 7.0 (no -full in the name) is also there (basically excluding most of the $SAGE_ROOT/src folder), but will still give the error:

OSError: [Errno 2] No such file or directory:
'/usr/lib/sagemath/src/build/cythonized/sage/structure/list_clone.c'

After that the debian/postinst will then try to run sage once, then stop, as not nearly enough paths were patched. Even removing relocate-once.py is not enough:

┌────────────────────────────────────────────────────────────────────┐
│ SageMath Version 7.0, Release Date: 2016-01-19                     │
│ Type "notebook()" for the browser-based notebook interface.        │
│ Type "help()" for help.                                            │
└────────────────────────────────────────────────────────────────────┘

ERROR:  The Sage installation tree has moved

from /mnt/highperf/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidma
  to /usr/lib/sagemath

This is not supported, and Sage will not work. To install Sage from a
binary package:

1. Open the .tar.bz2 archive (or .dmg on OSX)

2. Move the SageMath folder/app to where you want it to be. You can
   also rename the directory now.

3. Start sage for the first time. This will then automatically patch
   paths in binaries.

After starting Sage for the first time you cannot change the
installation any more. To install Sage elsewhere, start over from the
binary package. Or recompile Sage from scratch in the new location
("make distclean && make")

dpkg: error processing package sagemath-upstream-binary (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 sagemath-upstream-binary
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@alice-laptop:~#
root@alice-laptop:~# ls /usr/lib/sagemath/
COPYING.txt  README.txt  VERSION.txt  build  local  relocate-once.py  sage  src
root@alice-laptop:~# rm /usr/lib/sagemath/relocate-once.py
root@alice-laptop:~# sage
┌────────────────────────────────────────────────────────────────────┐
│ SageMath Version 7.0, Release Date: 2016-01-19                     │
│ Type "notebook()" for the browser-based notebook interface.        │
│ Type "help()" for help.                                            │
└────────────────────────────────────────────────────────────────────┘

ERROR:  The Sage installation tree has moved

from /mnt/highperf/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidma
  to /usr/lib/sagemath

This is not supported, and Sage will not work. To install Sage from a
binary package:

1. Open the .tar.bz2 archive (or .dmg on OSX)

2. Move the SageMath folder/app to where you want it to be. You can
   also rename the directory now.

3. Start sage for the first time. This will then automatically patch
   paths in binaries.

After starting Sage for the first time you cannot change the
installation any more. To install Sage elsewhere, start over from the
binary package. Or recompile Sage from scratch in the new location
("make distclean && make")

root@alice-laptop:~# apt-get install sagemath-upstream-binary-full

The workaround is to install sagemath-upstream-binary-full (and in 6.9 to ignore the error).

It is possible to install the 7.0 now from the dev PPA and then to disable it:

sudo apt-add-repository -y ppa:aims/sagemath-dev

sudo apt-get update

sudo apt-get install sagemath-upstream-binary-full

sudo mv /etc/apt/sources.list.d/aims-sagemath-dev-trusty.list{,.disabled}

Notes:

The plan is (probably) for 7.1 is to move the relocate-once.py out of the first run of sage during debian/postinst, and to do it in our autobuild.sh before it runs dpkg-buildpackage to prepare the source package for upload to the PPA. This way we can keep both the larger and smaller versions.

If you are going to do the unwise and unsupported thing of compiling optional packages as root on dpkg-owned directory, you will need the -full package. We plan to change the user owning the files so that even if we break debsums we don't have to compile as root to customize a PPA install.

I have no idea where to start, but perhaps there should be a ticket to make sage relocatable? Patching paths in binaries after compilation blows my mind.

Regards,

Jan

[HG]

Following the ppa sage link, I tried to install sage-7, it works fine. Time for patching, but no problem.
I use two version of sagemath the ppa and developer one (which I compile) and after I compile sagemanifolds (which I can't do at the moment with the ppa because it's installed system-wide).
Thanks for this good work
Henri

[Jan Groenewald]

Hi

On 4 February 2016 at 12:20, HG <henri....@gmail.com> wrote:

Following the ppa sage link, I tried to install sage-7, it works fine. Time for patching, but no problem.
I use two version of sagemath the ppa and developer one (which I compile) and after I compile sagemanifolds (which I can't do at the moment with the ppa because it's installed system-wide).
Thanks for this good work
Henri

If you run sagemath-upstream-binary-full it is technically possible to run sage manifolds. We had this deployed on campus last week. But you have to reinstall it after every upgrade. It is bad because:

1) you should not compile as root. There is a past bug where compiling some package would delete a system file [1]. Malicious code could hose or rootkit your system. We plan in future to have the sage tree owned by a sagemath user, which will alleviate that (only) if people remember to change to that user before customizing / adding optional packages and rebuilding.

2) bad practice since dpkg (or your distro package manager) is supposed to manage the system files, but it is technically possible.

It will not work with sagemath-upstream-binary (without -full) since the SAGE_ROOT/src folder is missing and many files you need for rebuilding sage.

I will try to release a 7.0.1 (rather than waiting for 7.1) which also fixes the smaller sagemath-upstream-binary in the PPAs. Note 7.0 is now in the stable PPA for 14.04.

Regards,

Jan

[Jan Groenewald]

A fixed version (you can run sagemath-upstream-binary without the -full) is now busy copying from the dev PPA to the stable PPA.

7.0~aimsppa1~qa201602060140061

Regards,

Jan