.configure stalls

[Frédéric Chapoton]

Hello,

after pulling 9.4.beta2, I got some problems with numpy compilation.

I thought I solved them in some ad-hoc way but then I started to get problems with pip and tornado.

In despair, I typed "make disclean" which I try hard to avoid usually, as my machine is very slow. But now, even .configure is no longer working. It reaches the following point and then stay there forever :

Checking whether SageMath should install SPKG pari...
checking whether any of gmp mpir readline is installed as or will be installed as SPKG... no
checking for gp... /usr/local/bin/gp
checking for gphelp... /usr/bin/gphelp
checking whether gphelp has access to the documentation... yes
checking is pari_elldata installed? ...

I am on ubuntu 20.04, and pari-elldata is installed.

I tried in a fresh clone of the sage repo, very same problem. I also removed my .sage.

Frédéric

[Dima Pasechnik]

Did you run ./bootstrap ?

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/d199241b-ddc3-4b4d-9f26-1fa67b4941f2n%40googlegroups.com.

[Dima Pasechnik]

On Sun, Jun 20, 2021 at 8:58 PM Frédéric Chapoton <fchap...@gmail.com> wrote:
>
> Hello,
>
> after pulling 9.4.beta2, I got some problems with numpy compilation.
> I thought I solved them in some ad-hoc way but then I started to get problems with pip and tornado.
>
> In despair, I typed "make disclean" which I try hard to avoid usually, as my machine is very slow. But now, even .configure is no longer working. It reaches the following point and then stay there forever :
>
> Checking whether SageMath should install SPKG pari...
> checking whether any of gmp mpir readline is installed as or will be installed as SPKG... no
> checking for gp... /usr/local/bin/gp
> checking for gphelp... /usr/bin/gphelp

in your PATH you have two different versions of pari installed, one in
/usr, and another in /usr/local

this wreaks havoc.

> checking whether gphelp has access to the documentation... yes
> checking is pari_elldata installed? ...
>
> I am on ubuntu 20.04, and pari-elldata is installed.
>
> I tried in a fresh clone of the sage repo, very same problem. I also removed my .sage.
>
> Frédéric
>

[Frédéric Chapoton]

Thanks a lot, Dima. Indeed. I removed the /usr/local/bin/gp and configure went smoothly. I think I have a similar problem with pip, but not sage-related.

Frederic

[Frédéric Chapoton]

Hello,

So now I am back at the numpy issue when building sagelib, that I sort of "fixed" before by a forced and ugly "sage -pip install numpy".

Is my computer just too old ? I add information about it at the end of my message.

make --no-print-directory sagelib-SAGE_VENV-no-deps

cd '/home/chapoton/sage/build/pkgs/sagelib' && . '/home/chapoton/sage/src/bin/sage-src-env-config' && . '/home/chapoton/sage/src/bin/sage-env-config' && . '/home/chapoton/sage/src/bin/sage-env' && . '/home/chapoton/sage/build/bin/sage-build-env-config' && . '/home/chapoton/sage/build/bin/sage-build-env' && SAGE_SPKG_WHEELS=/home/chapoton/sage/local/var/lib/sage/wheels SAGE_INST_LOCAL=/home/chapoton/sage/local sage-logger -p '/home/chapoton/sage/build/pkgs/sagelib/spkg-install' '/home/chapoton/sage/logs/pkgs/sagelib-9.4.beta2.log'

[sagelib-9.4.beta2] Discovering Python/Cython source code....

[sagelib-9.4.beta2] distributions = ['']

[sagelib-9.4.beta2] Discovered Python/Cython sources, time: 19.60 seconds.

[sagelib-9.4.beta2] running build

[sagelib-9.4.beta2] Generating auto-generated sources

[sagelib-9.4.beta2] Building interpreters for fast_callable

[sagelib-9.4.beta2] -> First build of interpreters

[sagelib-9.4.beta2] running build_cython

[sagelib-9.4.beta2] Enabling Cython debugging support

[sagelib-9.4.beta2] ************************************************************************

[sagelib-9.4.beta2] Traceback (most recent call last):

[sagelib-9.4.beta2]   File "setup.py", line 96, in <module>

[sagelib-9.4.beta2]     code = setup(

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup

[sagelib-9.4.beta2]     return distutils.core.setup(**attrs)

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 148, in setup

[sagelib-9.4.beta2]     dist.run_commands()

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands

[sagelib-9.4.beta2]     self.run_command(cmd)

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command

[sagelib-9.4.beta2]     cmd_obj.run()

[sagelib-9.4.beta2]   File "/home/chapoton/sage/build/pkgs/sagelib/src/sage_setup/command/sage_build.py", line 37, in run

[sagelib-9.4.beta2]     build.run(self)

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 135, in run

[sagelib-9.4.beta2]     self.run_command(cmd_name)

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command

[sagelib-9.4.beta2]     self.distribution.run_command(command)

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command

[sagelib-9.4.beta2]     cmd_obj.run()

[sagelib-9.4.beta2]   File "/home/chapoton/sage/build/pkgs/sagelib/src/sage_setup/command/sage_build_cython.py", line 211, in run

[sagelib-9.4.beta2]     self.sage_include_dirs = sage_include_directories(use_sources=True)

[sagelib-9.4.beta2]   File "/home/chapoton/sage/build/pkgs/sagelib/src/sage/env.py", line 376, in sage_include_directories

[sagelib-9.4.beta2]     import numpy

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/numpy/__init__.py", line 145, in <module>

[sagelib-9.4.beta2]     from . import core

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/numpy/core/__init__.py", line 22, in <module>

[sagelib-9.4.beta2]     from . import multiarray

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/numpy/core/multiarray.py", line 12, in <module>

[sagelib-9.4.beta2]     from . import overrides

[sagelib-9.4.beta2]   File "/home/chapoton/sage/local/lib/python3.8/site-packages/numpy/core/overrides.py", line 7, in <module>

[sagelib-9.4.beta2]     from numpy.core._multiarray_umath import (

[sagelib-9.4.beta2] RuntimeError: NumPy was built with baseline optimizations: 

[sagelib-9.4.beta2] (SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42) but your machine doesn't support:

[sagelib-9.4.beta2] (POPCNT).

[sagelib-9.4.beta2] ************************************************************************

[sagelib-9.4.beta2] Error building the Sage library

computer profile :

processor : 1

vendor_id : GenuineIntel

cpu family : 6

model : 23

model name : Intel(R) Core(TM)2 Duo CPU     P8600  @ 2.40GHz

stepping : 10

microcode : 0xa0b

cpu MHz : 1492.048

cache size : 3072 KB

physical id : 0

siblings : 2

core id : 1

cpu cores : 2

apicid : 1

initial apicid : 1

fpu : yes

fpu_exception : yes

cpuid level : 13

wp : yes

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl cpuid aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm pti tpr_shadow vnmi flexpriority dtherm ida

bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit

bogomips : 4788.38

[Dima Pasechnik]

On Mon, Jun 21, 2021 at 10:50 AM Frédéric Chapoton <fchap...@gmail.com> wrote:
>
> Hello,
>

we are seeing this error on much newer machines, too.
It's a subtle bug, I guess.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/763f8650-9803-4bba-a0ec-46744204fc22n%40googlegroups.com.

[Dima Pasechnik]

On the other hand, what did 
"sage -pip install numpy" actually do? Did it fetch a binary from the net?

Did it fetch source and built?

[Dima Pasechnik]

This is now https://trac.sagemath.org/ticket/32021

[Frédéric Chapoton]

I did it again (see below) and it allowed the compilation of sage to start.

~/sage$ sage -pip install --upgrade --force-reinstall numpy

Collecting numpy

  Downloading numpy-1.20.3-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.4 MB)

     |████████████████████████████████| 15.4 MB 2.9 MB/s 

Installing collected packages: numpy

  Attempting uninstall: numpy

    Found existing installation: numpy 1.20.3

    Uninstalling numpy-1.20.3:

      Successfully uninstalled numpy-1.20.3

Successfully installed numpy-1.20.3

[Dima Pasechnik]

On Mon, Jun 21, 2021 at 4:45 PM Frédéric Chapoton <fchap...@gmail.com> wrote:
>
> I did it again (see below) and it allowed the compilation of sage to start.
>
> ~/sage$ sage -pip install --upgrade --force-reinstall numpy
> Collecting numpy
> Downloading numpy-1.20.3-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.4 MB)
> |████████████████████████████████| 15.4 MB 2.9 MB/s
> Installing collected packages: numpy
> Attempting uninstall: numpy
> Found existing installation: numpy 1.20.3
> Uninstalling numpy-1.20.3:
> Successfully uninstalled numpy-1.20.3
> Successfully installed numpy-1.20.3

I don't know how these manylinux2010 wheels are meant to work, but
here obviously one either had it
built for the dumbest possible CPU (most probably not the case here)
or adjust the processor capabilities during the installation,
something I have no idea how it could work.

Does building sagelib error out just like before?

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/e12a8e58-03e5-42aa-87b6-1f21ea517580n%40googlegroups.com.

[Matthias Koeppe]

I don't think prebuilt numpy wheels can work reliably with our distribution. We would be linking to two copies of BLAS - one provided by numpy, one linked from our various other libraries.

Further discussion of this - https://trac.sagemath.org/ticket/31251 ("Meta-ticket: Distribution as wheels"). 

I would suggest not to try to diagnose the precise failure mode that come from combining the Sage distribution with prebuilt wheels from PyPI.

[Frédéric Chapoton]

This *very ugly hack* of pip-installing numpy allowed me to compile sage and get a seemingly working sage. I agree that this is in no way a satisfactory solution.

[Dima Pasechnik]

How exactly two copies of BLAS are harmful (provided that numpy always
uses its own, and it is not used by the other potential users)?
I imagine wheels mechanics is guaranteeing this - otherwise it will
break everything, not just Sage.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/f42fab98-5820-4764-918a-315b84a23417n%40googlegroups.com.