Error installing package numpy-1.19.1

[brettpim]

I am trying to install sage from the master branch on PC running Ubuntu 20.04.

The make is failing to build numpy-1.19.1.

It is complaining about not finding cblas or blas.

In my config.log it finds openblas and so decides it will not install is as SPKG

but when I run python3:

>>> import numpy

>>> numpy.show_config()

it reports

blas_info:
    libraries = ['blas', 'blas']
    library_dirs = ['/usr/lib/x86_64-linux-gnu']
    include_dirs = ['/usr/local/include', '/usr/include']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    libraries = ['blas', 'blas']
    library_dirs = ['/usr/lib/x86_64-linux-gnu']
    include_dirs = ['/usr/local/include', '/usr/include']
    language = c

which makes me think there is something wrong with blas.

[Dima Pasechnik]

looks like you have a copy of (c)blas in /usr/local - and this leads to trouble

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/a8054ce0-69ee-4537-80ed-f51ced0f7c8cn%40googlegroups.com.

[Dima Pasechnik]

On Sat, Feb 27, 2021 at 4:07 AM brettpim <br...@math.carleton.ca> wrote:
>
> /usr/local/ is where pip has installed my python packages so don't these files belong here?

>
>
> On Friday, February 26, 2021 at 5:20:03 PM UTC-6 dim...@gmail.com wrote:
>>
>> looks like you have a copy of (c)blas in /usr/local - and this leads to trouble

oops, sorry, it's a wrong guess, I suppose.

What is the branch you are installing? master from github, which is Sage 9.2 ?
Or the development branch, 9.3.beta7?

Was it an attempt to re-build after git pull,or a fresh install?
Fresh install should work, but for an update, you might need a serious clean-up:

make bootstrap-clean distclean

Sage has a bit fragile system of re-creating missing .pc files for
(c)blas/lapack, and
recent changes in the build system might have confused it (so that's
why a fresh rebuild
might be needed)

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/4c7f11e6-246d-4f58-b6d2-dfcbaf0b284bn%40googlegroups.com.

[brettpim]

DIma,

I found that pip installs in /usr/local when it is run as sudo.  I have not found anything definitive that says it should not be run as sudo but just in case I sudo pip uninstalled all the packages in /usr/local and pip installed them as a user which puts them all in $HOME/.local.  But even after this there something is still wrong with my numpy install and I am still working on.

While I am investigating my numpy problem I will try the serious clean-up you suggest.  My build was an attempt at a re-build after a git pull so the clean-up is justified.  I will report back

thanks

brett

[brettpim]

make bootstrap-clean distclean  did not solve the problem.  I have considered removing python3 from my system and reinstalling it but there are so many other things I have that depend on python3 and would be removed.  I don't want to have to reinstall such a huge list.  I am not sure what to do next.  One thought is to delete my entire sage directory and pull it afresh but I am not sure this will address the issue that blas is not being found properly.

[brettpim]

also my git branches I am working on would be lost if I wiped everything.

[brettpim]

I have finally tracked down and removed all installations of numpy; I also removed libopenblas.  I have done make bootstrap-clean distclean; ./configure; and started make again.  I will report the results

[Dima Pasechnik]

by right, one ought to read the last part of the output of ./configure
for advice on
packages to to install, and follow it.

Without libopenblas-dev installed you're going to build it, and
everything that depends on it, from source...

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/249a4e39-19bb-423b-8103-1501e5c691can%40googlegroups.com.

[brettpim]

True, and removing libopenblas did not help; I am still getting the same error.  I put libopenblas back but not numpy and am trying again

[brettpim]

No luck.  I am not sure what to do next.

[Dima Pasechnik]

this seems to be the error here:
https://trac.sagemath.org/ticket/31135

I wonder if you can run

make toolchain

and only then run

make

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/9551c544-abe6-44b2-ac55-e1884d0ad060n%40googlegroups.com.

[Matthias Koeppe]

Whether numpy is installed in the system or not does not matter for the Sage installation. Sage uses its own virtual environment without access to system Python packages.

Please switch to the latest development branch using "git checkout" and use "make distclean" so that the source tree is clean

Check that /usr/local/ does not contain libraries that shadow system libraries. In particular, check /usr/local/lib/pkgconfig

Then a new build using "make build" should work; if there is a new failure, please send the logs, including config.log

[brettpim]

Dima,

I ran

make toolchain

and then

make

This was successful.  I note in the log section on numpy that it is using a non-optimized netlib Blas.

What exactly does "make toolchain" do and not do?  Is there anything likely not to work with the sage that successfully built?

thanks

brett 

[brettpim]

Matthias,

I did have python libraries in /usr/local but I uninstalled them. Then I did a pull of most recent master branch followed by a 

make distclean

and

make

and I had the failure to build numpy in the detailed in the log attached to my Feb 27, 2021, 7:26:36 PM post.

regards

brett

[Matthias Koeppe]

post the logs please - config.log and the numpy.....log

[Dima Pasechnik]

On Tue, 2 Mar 2021, 03:11 brettpim, <br...@math.carleton.ca> wrote:

I ran

make toolchain

and then

make

This was successful.  I note in the log section on numpy that it is using a non-optimized netlib Blas.

OK, great. 

What exactly does "make toolchain" do and not do?  Is there anything likely not to work with the sage that successfully built?

in particular "make toolchain" readies 

pkg-config .pc files for blas, lapack, etc. 

(on a modern Linux system with most recommended packages installed it does not do much more)

numpy has a very old and a bit insane way to check for its dependencies (basically, compilers and (c)blas/lapack)

So somehow it manages to get itself confused on your machine, when we build it as Sage package.

Your logs say

Traceback (most recent call last):
  File "../lapack_conf.py", line 7, in <module>
    pc_blas   = pkgconfig.parse('cblas blas')
  File "/home/brett/Projects/SAGE/sage/local/lib/python3.8/site-packages/pkgconfig/pkgconfig.py", line 248, in parse
    _raise_if_not_exists(package)
  File "/home/brett/Projects/SAGE/sage/local/lib/python3.8/site-packages/pkgconfig/pkgconfig.py", line 103, in _raise_if_not_exists
    raise PackageNotFoundError(package)
pkgconfig.pkgconfig.PackageNotFoundError: cblas not found

which indicate that said pkg-config files are not present (at least one for cblas). They are recreated by the "make toolchain",

so that's why it all works after this. Your config.log indicates that cblas capabilities are available in your openblas,

so it boils down to .pc file not bein set up.

The reason you see netlib Blas is due to your openblas not recognised to have lapack capabilities,

as far as I can see from your config.log. 

Maybe an Ubuntu "feature", as it's perfectly possible to have openblas-optimised lapack included in libopenblas, maybe Ubuntu, unlike e.g. Debian, does not do it.

Maybe it's a bug in our tests, which are admittedly a hack, introduced in

https://trac.sagemath.org/ticket/29361

This hack is something that should go once we al last deside that we should not bother with packaging compilers

in Sage (FFS, this moment cannot be too soon, it should have happen years ago!), as this hack is meant to cover the case of no Fortran compiler available on the system (but I start ranting here, sorry :-)).

To test is your Sage is OK, you can run

   make ptest

to run the standard Sage tests, to see if nothing is broken (takes 2-3 hours on a modern PC)

Cheers

Dima

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/a206e982-f6b7-4d50-9334-7967c18df575n%40googlegroups.com.

[brettpim]

make ptest

reported "all tests passed!"

thank you for your help getting this working.

brett