CentOS binary (via binary-pkg)

[Franco Saliola]

Hello,

I am trying to create a binary for CentOS 7. I have not been able to find a guide on how to create binaries, so I've been piecing together the procedure from various sources and so I have a few questions.

First of all, my goal is to use sage on a cluster running CentOS 7. I can't install the dependencies to compile sage, so I thought of creating a CentOS binary on a virtual machine and transferring the resulting binary to the cluster. Any ideas or pointers on how to get sage running in this situation would be most welcome!

As for creating the binary, here is what I have done so far:

- I created a VM with a minimal CentOS 7 installation and installed the dependencies listed in the SageMath installation guide.

- I cloned sagemath/binary-pkg.

- I changed the branch in sage.yaml from develop to master.

- I executed: make bdist-sage-linux

Eventually, it failed with a "No space left on device" error. Is it expected that this process would take more than 20GB of disk space? If so, then how much disk space do I need?

Best,

Franco

[Isuru Fernando]

Have you looked at installing sage from conda? It's a package manager that can be installed without root privileges and you don't have to compile sage because it's already compiled.

https://doc.sagemath.org/html/en/installation/conda.html

Isuru

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/7792aae0-c0f8-4bc5-bf90-612f9bf91d89o%40googlegroups.com.

[Franco Saliola]

Hi Isuru,

Thank you for the suggestion. I looked into this, and although it looks intriguing, there seems to be a site policy of avoiding conda on the cluster I am trying to use.

I think I will post the question about binary-pkg to the github issues page for that project.

Best,

Franco

You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/nftiNNR_5cE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CA%2B01voNqpdLF3sV0L%2BCGF_q_C-PMo4E2skkuCa0X7W9eOey4NA%40mail.gmail.com.

[Isuru Fernando]

Hi,

You can install conda on your local machine, make a new sage environment using conda and then pack up that environment using `conda-pack` and then distribute the sage environment to the cluster. This way, you will only have sage distributed and no conda-pack. This is similar to what you would do with binary-pkg.

Btw, do you know what the issue is with using conda?

Isuru

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CALQzqJctGXcnb6Epsk25vo1b8tP8CjLjGGTJTG1gxdi4K12cnQ%40mail.gmail.com.

[Franco Saliola]

Btw, do you know what the issue is with using conda?

I'm new to using these specific machines, but the link below says to avoid anaconda. It also contains a section on transitioning from conda to virtualenv, which I took to mean that conda is also frowned upon. But maybe they just mean anaconda specifically.

https://docs.computecanada.ca/wiki/Anaconda/en#Do_not_install_Anaconda_on_our_clusters

Thanks again!

Franco

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CA%2B01voNTDE4_7dyidad_-vdsNwHYa6AnxXoronvyQmzKN%2Bh-%2BQ%40mail.gmail.com.

[Dima Pasechnik]

On Mon, 22 Jun 2020, 20:30 Franco Saliola, <sal...@gmail.com> wrote:

Btw, do you know what the issue is with using conda?

I'm new to using these specific machines, but the link below says to avoid anaconda. It also contains a section on transitioning from conda to virtualenv, which I took to mean that conda is also frowned upon. But maybe they just mean anaconda specifically.

https://docs.computecanada.ca/wiki/Anaconda/en#Do_not_install_Anaconda_on_our_clusters

these reasons sound like FUD to me.

(they don't read like outright prohibition either)

Besides, I don't understand you saying you can't get tools to build Sage there - if they tell you to avoid Conda they should either provide you an equivalent, or stop being silly.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CALQzqJf3DzONw-c3NpZW5b6BbF1Dwsw7r11fRqgKconduXWNmw%40mail.gmail.com.

[Franco Saliola]

On Mon, Jun 22, 2020 at 3:47 PM Dima Pasechnik <dim...@gmail.com> wrote:

On Mon, 22 Jun 2020, 20:30 Franco Saliola, <sal...@gmail.com> wrote:

Btw, do you know what the issue is with using conda?

I'm new to using these specific machines, but the link below says to avoid anaconda. It also contains a section on transitioning from conda to virtualenv, which I took to mean that conda is also frowned upon. But maybe they just mean anaconda specifically.

https://docs.computecanada.ca/wiki/Anaconda/en#Do_not_install_Anaconda_on_our_clusters

these reasons sound like FUD to me.

(they don't read like outright prohibition either)

Besides, I don't understand you saying you can't get tools to build Sage there - if they tell you to avoid Conda they should either provide you an equivalent, or stop being silly.

:)

But they are being helpful in other ways. They already had Sage 8.1 available and they created an installation ticket for Sage 9.1. But I know that I will eventually want to run a beta or patched version, so I'm trying to figure out how to do this.

Franco

[Isuru Fernando]

That document seems to be written by a person who only had a cursory look at conda without realizing that there are options that you could use to avoid these issues.

As Dima said, if they want you to install from source, they should provide a compiler and other tools that you can use to compile sage.

I've added some responses below.

Isuru

> Anaconda very often installs software (compilers, scientific libraries etc.) which already exist on Compute Canada clusters as modules, with a configuration that is not optimal.

Given that they allow singularity, this is moot. binary-pkg, python wheels does the same thing.

> It installs binaries which are not optimized for the processor architecture on our clusters.

Sure. But if you are going to use binary-pkg, you'll get the same issue. Python wheels have the same issue.

> It makes incorrect assumptions about the location of various system libraries.

I don't know what this is referring to.

> Anaconda uses the $HOME directory for its installation, where it writes an enormous number of files.
> A single Anaconda installation can easily absorb almost half of your quota for the number of files in your home directory.

Same issue with singularity and binary-pkg. You can install conda to some other place.

> Anaconda is slower than the installation of packages via Python wheels.

Okay. I don't know why this is a huge problem. Most of your time is spent on calculations anyway. Installation time is a tiny fraction.

> Anaconda modifies the $HOME/.bashrc file, which can easily cause conflicts.

You are asked about this in the command line and can opt out.

> A conda recipe forces the installation of R. This installation does not perform nearly as well as the version we provide as a module (which uses Intel MKL). This same R does not work well, and jobs launched with it may die and waste both computing resources as well as your time.

You can use R with MKL from conda-forge. You can even use Sage with MKL on conda-forge. (You can't use MKL with python wheels or binary-pkg)