Install of sage on macOS

[Mathieu Dutour]

I would like to install SAGE on a macOS MacBook Pro16 computer.

The problem I have is that Sage is killed when trying to install it

by the system. The reasons is laudable: Do not run untrusted binaries

on a macintosh.

But in that case, it makes installation kind of impossible. The reason

is that the first binary to be allowed on "Security & Privacy" is Python3.8

If the story stopped here, then that would be perfectly fine. However, after

I allow the Python3.8 I have further libraries to allow and it does not appear

to converge as I keep allowing the same libraries over and over.

I tried two ways of installing:

1) Downloading the .tar.bz2 and installing it in my $HOME/opt directory

just as I do on Linux.

2) Downloading the .dmg archive and installing sage in /Application directory.

I would have expected that it worked since /Application is a root directory

and not user one. Also the .dmg is a package format so should allow

simple authorization scheme.

I followed the "Sage Installation Guide. Release 9.2" and unfortunately

that problem does not appear to be addressed there.

  Mathieu

[Dima Pasechnik]

Hi Mathieu,

On Wed, Dec 2, 2020 at 6:54 PM Mathieu Dutour <mathieu...@gmail.com> wrote:
>
> I would like to install SAGE on a macOS MacBook Pro16 computer.

Building binary installers on macOS has become a dark art, we don't
have people who
master it well any more :-(

The quickest way is to use Conda.
https://doc.sagemath.org/html/en/installation/conda.html

If you use Homebrew then building from source is a reasonably quick option,
too.
Building from source is possible with just XCode installed, but this takes long
time, as it would build gcc/gfortran etc etc

HTH
Dima

> --
> You received this message because you are subscribed to the Google Groups "sage-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-support...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/819b6b90-2cae-4829-9dbb-cd957d8bccdan%40googlegroups.com.

[Dima Pasechnik]

On Wed, Dec 2, 2020 at 7:10 PM Dima Pasechnik <dim...@gmail.com> wrote:
>
> Hi Mathieu,
>
> On Wed, Dec 2, 2020 at 6:54 PM Mathieu Dutour <mathieu...@gmail.com> wrote:
> >
> > I would like to install SAGE on a macOS MacBook Pro16 computer.
>
> Building binary installers on macOS has become a dark art, we don't
> have people who
> master it well any more :-(
>
> The quickest way is to use Conda.
> https://doc.sagemath.org/html/en/installation/conda.html
>
> If you use Homebrew then building from source is a reasonably quick option,
> too.
> Building from source is possible with just XCode installed, but this takes long
> time, as it would build gcc/gfortran etc etc

PS. If you're already on macOS 11, then it might be problematic to
build Sage 9.2 there, as
it was not tested on the official released version.
In this case you're advised to use the latest beta for SageMath 9.3

[Mathieu Dutour]

Building binary installers on macOS has become a dark art, we don't
have people who
master it well any more :-(

What a pity. What are the scripts for the creation of the binaries?

Maybe that can be addressed easily.

The quickest way is to use Conda.
https://doc.sagemath.org/html/en/installation/conda.html

Yes, I use it for other packages.

I wanted to avoid that route as it has its own problems.

 

If you use Homebrew then building from source is a reasonably quick option,
too.

Ok, will try that then.

Building from source is possible with just XCode installed, but this takes long
time, as it would build gcc/gfortran etc etc

Thanks for info.

  Mathieu

[Dima Pasechnik]

On Wed, Dec 2, 2020 at 7:57 PM Mathieu Dutour <mathieu...@gmail.com> wrote:
>>
>> Building binary installers on macOS has become a dark art, we don't
>> have people who
>> master it well any more :-(
>
> What a pity. What are the scripts for the creation of the binaries?
> Maybe that can be addressed easily.

The script is in https://github.com/sagemath/binary-pkg
A relocatable binary is made, so that the hardcoded paths in binaries
(i.e. dylibs and executables) are rewritten by
an installation script with the actual values, before the 1st run..

I don't see how this can be reconciled with the need to codesign
binaries on modern macOS.
One needs an installer that will build something that must be
installed in /Applications
and codesign the binaries before the distribution is wrapped up.
It is probably not much work for someone who knows these macOS details, though.

We're removed an even more broken thing, a macOS app, in a recent sage 9.3 beta.
https://trac.sagemath.org/ticket/30867
It's even more broken, and requires work in Xcode IDE to be fixed.
(not sure whether this is a worthwhile job)

>
>> The quickest way is to use Conda.
>> https://doc.sagemath.org/html/en/installation/conda.html
>
> Yes, I use it for other packages.
> I wanted to avoid that route as it has its own problems.

I wonder what problems you have (we have a Conda expert helping
package Sage on Conda, he might be interested to know.

Perhaps you can have a separate Conda install, if nothing else...

>
>>
>> If you use Homebrew then building from source is a reasonably quick option,
>> too.
>
> Ok, will try that then.
>
>> Building from source is possible with just XCode installed, but this takes long
>> time, as it would build gcc/gfortran etc etc
>
> Thanks for info.
>
> Mathieu
>

> --
> You received this message because you are subscribed to the Google Groups "sage-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-support...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/bcdfa092-3fd3-4e57-8e87-46c96d7ad53an%40googlegroups.com.

[john_perry_usm]

Have you tried this?

   sudo xattr -r -d com.apple.quarantine <path/to/Sage>

It removes the "quarantine" signal on files. I used it to get Sage running on Catalina. I've had to use this with some other things, too. I learned of it through one of the StackOverflow sites, and it may also be in Sage's documentation somewhere.

john perry

[Mathieu Dutour]

Thank you for providing this. Keep that in mind for future.

As it turns out, the solution to installing was incredibly simple:

"brew install sage"

I think this should be mentioned on the "Sage Installation Guide"

as this is a very convenient and standard procedure on Macintosh.

Your mention of "sudo xattr ..." should be mentioned as well

I think.

  Mathieu

[Dima Pasechnik]

On Thu, Dec 3, 2020 at 5:39 PM Mathieu Dutour <mathieu...@gmail.com> wrote:

Thank you for providing this. Keep that in mind for future.

As it turns out, the solution to installing was incredibly simple:

"brew install sage"

on macOS 10.15.7 this appears to install the version 9.2 of  x86_64.app.dmg by

pulling it from a Sage mirror.

Does Sage installed this way work?

(I'd be surprised it was the case)

--

You received this message because you are subscribed to the Google Groups "sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-support...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/e6cb3fd4-f338-4fd6-bd48-f4d6ad7c7345n%40googlegroups.com.

[slelievre]

A ticket to track providing an app for macOS again:

- Provide macOS app again
  https://trac.sagemath.org/ticket/31000

[Dima Pasechnik]

On Thu, Dec 3, 2020 at 5:39 PM Mathieu Dutour <mathieu...@gmail.com> wrote:
>

> Thank you for providing this. Keep that in mind for future.
>
> As it turns out, the solution to installing was incredibly simple:
> "brew install sage"
>

dima@oucl13243 sage % brew install sage
==> Searching for similarly named formulae...
This similarly named formula was found:
imessage-ruby
To install it, run:
brew install imessage-ruby
Error: No available formula or cask with the name "sage".
==> Searching for a previously deleted formula (in the last month)...
Error: No previously deleted formula found.
==> Searching taps on GitHub...
Error: No formulae found in taps.

dima@oucl13243 sage % brew install --cask sage
Error: Cask 'sage' is unavailable: No Cask with this name exists.

> I think this should be mentioned on the "Sage Installation Guide"
> as this is a very convenient and standard procedure on Macintosh.
>

doesn't even install for me, see above.

[G. M.-S.]

Hi Dima.

As you said, "brew install sage" does nothing more than downloading a sage-*.app.dmg and copying it to /Applications, at least on an older OS.

Details follow for macOS High Sierra (10.13.6).  Yes, I know it is an "obsolete" version.

$ brew info sage

sage: 9.1,10.11.6

https://www.sagemath.org/

Not installed

From: https://github.com/Homebrew/homebrew-cask/blob/HEAD/Casks/sage.rb

==> Name

Sage

==> Description

None

==> Artifacts

SageMath-9.1.app (App)

/Applications/SageMath-9.1.app/Contents/Resources/sage/sage (Binary)

==> Analytics

install: 261 (30 days), 529 (90 days), 1,740 (365 days)

$ 

One can see in the file

https://github.com/Homebrew/homebrew-cask/blob/HEAD/Casks/sage.rb

that it downloads

https://mirrors.mit.edu/sage/osx/intel/sage-9.1-OSX_10.11.6-x86_64.app.dmg

or

https://mirrors.mit.edu/sage/osx/intel/sage-9.2-OSX_10.15.7-x86_64.app.dmg

(which will not work for recent OS).

I tried to install it:

$ brew install sage

[…]

==> Downloading https://mirrors.mit.edu/sage/osx/intel/sage-9.1-OSX_10.11.6-x86_64.app.dmg

######################################################################## 100.0%

==> Installing Cask sage

==> Creating Caskroom at /usr/local/Caskroom

==> We'll set permissions properly so we won't need sudo in the future.

Password:

==> Moving App 'SageMath-9.1.app' to '/Applications/SageMath-9.1.app'.

==> Linking Binary 'sage' to '/usr/local/bin/sage'.

🍺  sage was successfully installed!

$ 

But it is quarantined:

$ ls -al@ /Applications/SageMath-9.1.app

total 0

drwxr-xr-x@   4 128 May 23  2020 .

com.apple.quarantine   67 

drwxr-xr-x@   6 192 May 23  2020 Contents

com.apple.quarantine   67 

lrwxr-xr-x@   1  28 Dec  4 01:39 sage -> Contents/Resources/sage/sage

com.apple.quarantine   67 

$ 

and you get the classical

“SageMath-9.1.app” can’t be opened because the identity of the developer cannot be confirmed.

Opening it anyway, it is exactly the same as downloading the .app.dmg yourself (which I had already done).

I do not know if Homebrew does something more on a more recent OS.

Guillermo

P.S. After some trial and error, I have managed to install sage-9.2-OSX_10.15.7-x86_64.app.dmg on macOS 10.13.6, but I would be unable to say what made it work.

[Mathieu Dutour]

On Thursday, 3 December 2020 at 19:13:28 UTC+1 dim...@gmail.com wrote:

On Thu, Dec 3, 2020 at 5:39 PM Mathieu Dutour <mathieu...@gmail.com> wrote:

Thank you for providing this. Keep that in mind for future.

As it turns out, the solution to installing was incredibly simple:

"brew install sage"

on macOS 10.15.7 this appears to install the version 9.2 of  x86_64.app.dmg by

pulling it from a Sage mirror.

Does Sage installed this way work?

(I'd be surprised it was the case)

That is correct.

However, I had to give authorization only one time which represent

a progress and allowed me to run sage on macintosh.

I still think it should be mentioned in the Sage Installation Guide.

  Mathieu

[Dima Pasechnik]

as I already mentioned, for some reason this does not work on a macOS
10.15.7 machine I have access to.

Strangely enough, I needed to run first a command

brew cask

(don't know what it actually does, it did
download (probably git-cloned) something

==> Tapping homebrew/cask
Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask'...
remote: Enumerating objects: 55, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 574981 (delta 26), reused 19 (delta 14), pack-reused 574926
Receiving objects: 100% (574981/574981), 240.80 MiB | 7.59 MiB/s, done.
Resolving deltas: 100% (409134/409134), done.
Tapped 1 command and 3739 casks (3,857 files, 259MB).



brew cask

after which

brew install --cask sage

did seem to work.

Starting sage brought up the popup:

----------------------------------
“SageMath-9.2” cannot be opened because the developer cannot be verified.
macOS cannot verify that this app is free from malware.
---------------
[move to bin] [cancel]
-------------------------------------

So I did

% sudo xattr -r -d com.apple.quarantine /Applications/SageMath-9.2.app
xattr: No such file:
/Applications/SageMath-9.2.app/Contents/Resources/sage/local/share/jupyter/kernels/sagemath/doc

Apparently there is a dangling link pointing there, but OK. It has done the job,
and have an apparently working sage in terminal. Jupyter is broken:

% sage -n

┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 9.2, Release Date: 2020-10-24 │
│ Using Python 3.8.5. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
Please wait while the Sage Jupyter Notebook server starts...
Traceback (most recent call last):
File "/Applications/SageMath-9.2.app/Contents/Resources/sage/local/lib/python3.8/site-packages/sage/repl/ipython_kernel/install.py",
line 307, in have_prerequisites
from notebook.notebookapp import NotebookApp
File "/Applications/SageMath-9.2.app/Contents/Resources/sage/local/lib/python3.8/site-packages/notebook/notebookapp.py",
line 66, in <module>
from tornado import httpserver
File "/Applications/SageMath-9.2.app/Contents/Resources/sage/local/lib/python3.8/site-packages/tornado/httpserver.py",
line 29, in <module>
import ssl
File "/Applications/SageMath-9.2.app/Contents/Resources/sage/local/lib/python3.8/ssl.py",
line 98, in <module>
import _ssl # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
The Jupyter notebook requires ssl, even if you do not use
https. Install the openssl development packages in your system and
then rebuild Python (sage -f python3).


Also, the app in /Applications/SageMath-9.2

does not quite work, most stuff from the menu's does not launch, etc.

So this is not really working well, still. :-(

>
> Mathieu
>
> --
> You received this message because you are subscribed to the Google Groups "sage-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-support...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/4cafb667-ca96-4417-b57e-c3341bafb041n%40googlegroups.com.

[Mouse Mousevich]

A lot of people have a lot of problems with Sage-9.2 on MacOS, installing pre-compiled or building from the source.

Sage-9.2 has several idiosyncrasies that for me are hard to explain. One - it deliberately refuses to work with Macports-installed packages, even though their main difference from similar ones installed via Brew is their location. How crazy is that?

Second, pplpy-0.8.4 is "obtuse" enough to try compiling C++ file linear_algebra.cpp with clang instead of clang++, and then complain that CFLAGS that are (surprise!) C-specific, do not make sense for C++. Need I mention that it is impossible to locate clang but not clang++?

Now, from generic criticisms to specific errors and logs.

Platform

- iMac 2020, Intel Core i9 CPU, 32GM RAM

- macOS Catalina 10.15.7

- Xcode-12.3 (with command-line tools installed) in /Applications/Xcode.app, and separately installed CLT 12.3 in /Library/Developer/CommandLineTools.

- Macports 2.6.4, with a ton of packages installed in /opt/local, which include clang-11, gcc-10, python38 and python39, and OpenSSL-1.1.1i.

Problems

1. Sage-9.2 config refuses to use already-installed packages in /opt/local, spends sh*tload of time and bandwidth downloading what's already there (except that it downloads Python-3.8.5, and my machine has Python-3.8.6), and finally fails to compile that pile:
2. pplpy-0.8.4 uses clang (not clang++? Why?) to compile ppl/linear_algebra.cpp files, and fails because on my machine CFLAGS differ from CXXFLAGS (surprise! one sets -std=gnu18, and other one sets -std=gnu++17). This bug should manifest on all the .cpp files.

Logs

The main log is 5.7 MB, so I don't want to post it unless asked to. Excerpts:

Error building Sage.

The following package(s) may have failed to build (not necessarily

during this run of 'make all-build'):

* package:         pplpy-0.8.4

  last build time: Dec 20 22:51

  log file:        /Applications/SageMath/logs/pkgs/pplpy-0.8.4.log

  build directory: /Applications/SageMath/local/var/tmp/sage/build/pplpy-0.8.4 

and more specifically:

[pplpy-0.8.4]   creating build/temp.macosx-10.9-x86_64-3.8/ppl

[pplpy-0.8.4]   clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -O3 -std=gnu18 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -O3 -std=gnu18 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -Ippl -I/Applications/SageMath/local/lib/python3.8/site-packages/gmpy2 -Ippl -I -I/Applications/SageMath/local -I/Applications/SageMath/local/lib/python38.zip -I/Applications/SageMath/local/lib/python3.8 -I/Applications/SageMath/local/lib/python3.8/lib-dynload -I/Applications/SageMath/local/lib/python3.8/site-packages -I/private/var/folders/_l/4q83bg9j5ysb7qd1n9xpnb4h0000gn/T/pip-req-build-_4gwq_tw -I/Applications/SageMath/local/include/python3.8 -c ppl/linear_algebra.cpp -o build/temp.macosx-10.9-x86_64-3.8/ppl/linear_algebra.o

[pplpy-0.8.4]   error: invalid argument '-std=gnu18' not allowed with 'C++'

[pplpy-0.8.4]   error: command 'clang' failed with exit status 1

[pplpy-0.8.4]   Building wheel for pplpy (setup.py): finished with status 'error'

[pplpy-0.8.4]   ERROR: Failed building wheel for pplpy

BTW, why is it creating build for macosx-10.9 when the current version of the OS is 10.15, and SDK is for 11.1?

[Mouse Mousevich]

After changing the env compiler flags to stop pplpy-0.8.4 from confusing itself, and re-starting the build, it fails in scipy-1.5.2:

[scipy-1.5.2]   clang -bundle -undefined dynamic_lookup -L/Applications/SageMath/local/lib -Wl,-rpath,/Applications/SageMath/local/lib -L. -L/Applications/SageMath/local/lib -Wl,-rpath,/Applications/SageMath/local/lib -bundle -undefined dynamic_lookup -L/Applications/SageMath/local/lib -Wl,-rpath,/Applications/SageMath/local/lib -D__ACCELERATE__ build/temp.macosx-10.9-x86_64-3.8/scipy/fftpack/convolve.o -L/Applications/SageMath/local/lib -Lbuild/temp.macosx-10.9-x86_64-3.8 -o build/lib.macosx-10.9-x86_64-3.8/scipy/fftpack/convolve.cpython-38-darwin.so

[scipy-1.5.2]   building 'scipy.integrate._quadpack' extension

[scipy-1.5.2]   compiling C sources

[scipy-1.5.2]   C compiler: clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -O3 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -D__ACCELERATE__

[scipy-1.5.2] 

[scipy-1.5.2]   compile options: '-DHAVE_CBLAS -Iscipy/_lib/src -I/Applications/SageMath/local/lib/python3.8/site-packages/numpy/core/include -I/Applications/SageMath/local/include/python3.8 -c'

[scipy-1.5.2]   clang: scipy/integrate/_quadpackmodule.c

[scipy-1.5.2]   In file included from scipy/integrate/_quadpackmodule.c:4:

[scipy-1.5.2]   In file included from scipy/integrate/__quadpack.h:28:

[scipy-1.5.2]   In file included from /Applications/SageMath/local/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4:

[scipy-1.5.2]   In file included from /Applications/SageMath/local/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:

[scipy-1.5.2]   In file included from /Applications/SageMath/local/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822:

[scipy-1.5.2]   /Applications/SageMath/local/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with "          "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]

[scipy-1.5.2]   #warning "Using deprecated NumPy API, disable it with " \

[scipy-1.5.2]    ^

[scipy-1.5.2]   1 warning generated.

[scipy-1.5.2]   /Applications/SageMath/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Applications/SageMath/local/lib -Wl,-rpath,/Applications/SageMath/local/lib build/temp.macosx-10.9-x86_64-3.8/scipy/integrate/_quadpackmodule.o -L/Applications/SageMath/local/lib -L/Applications/SageMath/local/lib/gcc/x86_64-apple-darwin19.6.0/9.2.0 -L/Applications/SageMath/local/lib -L/Applications/SageMath/local/lib -L/Applications/SageMath/local/lib -Lbuild/temp.macosx-10.9-x86_64-3.8 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-10.9-x86_64-3.8/scipy/integrate/_quadpack.cpython-38-darwin.so

[scipy-1.5.2]   ld: library not found for -lSystem

[scipy-1.5.2]   collect2: error: ld returned 1 exit status

[scipy-1.5.2]   Running from SciPy source directory.

[scipy-1.5.2]   /Applications/SageMath/local/lib/python3.8/site-packages/numpy/distutils/system_info.py:838: UserWarning: Specified path /usr/local/include/python3.8 is invalid.

[scipy-1.5.2]     return self.get_paths(self.section, key)

[scipy-1.5.2]   /Applications/SageMath/local/lib/python3.8/site-packages/numpy/distutils/system_info.py:838: UserWarning: Specified path /opt/local/include/python3.8 is invalid.

[scipy-1.5.2]     return self.get_paths(self.section, key)

[scipy-1.5.2]   error: Command "/Applications/SageMath/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Applications/SageMath/local/lib -Wl,-rpath,/Applications/SageMath/local/lib build/temp.macosx-10.9-x86_64-3.8/scipy/integrate/_quadpackmodule.o -L/Applications/SageMath/local/lib -L/Applications/SageMath/local/lib/gcc/x86_64-apple-darwin19.6.0/9.2.0 -L/Applications/SageMath/local/lib -L/Applications/SageMath/local/lib -L/Applications/SageMath/local/lib -Lbuild/temp.macosx-10.9-x86_64-3.8 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-10.9-x86_64-3.8/scipy/integrate/_quadpack.cpython-38-darwin.so" failed with exit status 1

[scipy-1.5.2]   Building wheel for scipy (PEP 517): finished with status 'error'

[scipy-1.5.2]   ERROR: Failed building wheel for scipy

[scipy-1.5.2] Failed to build scipy

[scipy-1.5.2] ERROR: Failed to build one or more wheels

[scipy-1.5.2] Exception information:

[scipy-1.5.2] Traceback (most recent call last):

[scipy-1.5.2]   File "/Applications/SageMath/local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 228, in _main

[scipy-1.5.2]     status = self.run(options, args)

[scipy-1.5.2]   File "/Applications/SageMath/local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper

[scipy-1.5.2]     return func(self, options, args)

[scipy-1.5.2]   File "/Applications/SageMath/local/lib/python3.8/site-packages/pip/_internal/commands/wheel.py", line 184, in run

[scipy-1.5.2]     raise CommandError(

[scipy-1.5.2] pip._internal.exceptions.CommandError: Failed to build one or more wheels

[scipy-1.5.2] Removed build tracker: '/private/var/folders/_l/4q83bg9j5ysb7qd1n9xpnb4h0000gn/T/pip-req-tracker-ybfte_50'

[scipy-1.5.2] ********************************************************************************

[scipy-1.5.2] Error building a wheel for scipy-1.5.2

[Dima Pasechnik]

TL;DR.

patches are most welcome :-)

Details below.

On Wed, 23 Dec 2020, 02:40 Mouse Mousevich, <u...@ll.mit.edu> wrote:

A lot of people have a lot of problems with Sage-9.2 on MacOS, installing pre-compiled or building from the source.

Sage-9.2 has several idiosyncrasies that for me are hard to explain. One - it deliberately refuses to work with Macports-installed packages, even though their main difference from similar ones installed via Brew is their location. How crazy is that?

To me, it is macOS that has idiosyncrasies ;-)

But OK, SageMath problems here are

not deliberate, and caused by severe labour shortage, if you like.

We don't have a single MacPort user among a handful of SageMath developers. Please feel free to provide patches. (Yes, it means supporting yet another nonstandard location, more or less, I suppose)

Second, pplpy-0.8.4 is "obtuse" enough to try compiling C++ file linear_algebra.cpp with clang instead of clang++, and then complain that CFLAGS that are (surprise!) C-specific, do not make sense for C++. Need I mention that it is impossible to locate clang but not clang++?

pplpy upstream is here, it has a very busy maintainer who does not use macOS

https://gitlab.com/videlec/pplpy

- the main developer no longer works on SageMath, as far as I know.

patches are most welcome!

Now, from generic criticisms to specific errors and logs.

Platform

- iMac 2020, Intel Core i9 CPU, 32GM RAM

- macOS Catalina 10.15.7

- Xcode-12.3 (with command-line tools installed) in /Applications/Xcode.app, and separately installed CLT 12.3 in /Library/Developer/CommandLineTools.

XCode is a moving target, 9.2 was released before XCode 12.3

- Macports 2.6.4, with a ton of packages installed in /opt/local, which include clang-11, gcc-10, python38 and python39, and OpenSSL-1.1.1i.

one ought to use XCode's clang(++), and only need gfortran from the gcc package.

Fortran binary location can be explicitly provided by setting FC environment variable.

(same for C and C++, one can use CC and CXX)

If you like to enable MacPorts, think about providing an analog of

.homebrew-build-env script.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/63642cb1-a380-4570-b703-e6a873269adbn%40googlegroups.com.

[Mouse Mousevich]

Please see below.

TL;DR.

patches are most welcome :-)

Understood.

 

Sage-9.2 has several idiosyncrasies that for me are hard to explain. One - it deliberately refuses to work with Macports-installed packages, even though their main difference from similar ones installed via Brew is their location. How crazy is that?

To me, it is macOS that has idiosyncrasies ;-)

True ;-)

But still, my Macports point stands.

 

We don't have a single MacPort user among a handful of SageMath developers. Please feel free to provide patches. (Yes, it means supporting yet another nonstandard location, more or less, I suppose)

The fun part is that to support Macports, you need to look in /opt/local/include for header files, in /opt/local/lib for libraries, and in /opt/local/bin for executables. How hard does that sound?

And I daresay, that adding those directories to the Sage config code where it is currently looking for stuff, is probably easier for a Sage developer unfamiliar with Macports, than for a person experienced with Macports that has no clue about Sage guts (and isn't eager to dive into those! ;).

Second, pplpy-0.8.4 is "obtuse" enough to try compiling C++ file linear_algebra.cpp with clang instead of clang++, and then complain that CFLAGS that are (surprise!) C-specific, do not make sense for C++. Need I mention that it is impossible to locate clang but not clang++?

pplpy upstream is here, it has a very busy maintainer who does not use macOS

https://gitlab.com/videlec/pplpy

- the main developer no longer works on SageMath, as far as I know.

patches are most welcome!

I'll take a look later, but the likelihood of me being able to fix it isn't great.

 

Platform

- iMac 2020, Intel Core i9 CPU, 32GM RAM

- macOS Catalina 10.15.7

- Xcode-12.3 (with command-line tools installed) in /Applications/Xcode.app, and separately installed CLT 12.3 in /Library/Developer/CommandLineTools.

XCode is a moving target, 9.2 was released before XCode 12.3

True, but not really relevant in the context. The big difference/jump came with Xcode-10. Xcode-11 and -12 exacerbated that a bit, but I daresay, imperceptibly so.

 

- Macports 2.6.4, with a ton of packages installed in /opt/local, which include clang-11, gcc-10, python38 and python39, and OpenSSL-1.1.1i.

one ought to use XCode's clang(++), and only need gfortran from the gcc package.

So one does. ;-)

As you can see from the log, the build uses Xcode's clang and clang++. 

For my stuff, especially when I need to deal with sanitizers extensively, I use Macports clang. And, of course, Xcode's GCC really sucks, so installing a "real" one from Macports was a-must.

 

Fortran binary location can be explicitly provided by setting FC environment variable.

(same for C and C++, one can use CC and CXX)

It's all done in my config - except for Fortran. I don't use it any more (for the last 20 years, or so ;), but it's no problem installing it. Assuming Sage would be smart enough to use it (rather than downloading and building it's own GCC and GFortran)!

 

If you like to enable MacPorts, think about providing an analog of .homebrew-build-env script.

That's something constructive that I can probably do. Thanks for bringing it up. 

[Dima Pasechnik]

On Wed, 23 Dec 2020, 11:01 Mouse Mousevich, <u...@ll.mit.edu> wrote:

Please see below.

TL;DR.

patches are most welcome :-)

Understood.

 

Sage-9.2 has several idiosyncrasies that for me are hard to explain. One - it deliberately refuses to work with Macports-installed packages, even though their main difference from similar ones installed via Brew is their location. How crazy is that?

To me, it is macOS that has idiosyncrasies ;-)

True ;-)

But still, my Macports point stands.

 

We don't have a single MacPort user among a handful of SageMath developers. Please feel free to provide patches. (Yes, it means supporting yet another nonstandard location, more or less, I suppose)

The fun part is that to support Macports, you need to look in /opt/local/include for header files, in /opt/local/lib for libraries, and in /opt/local/bin for executables. How hard does that sound?

And I daresay, that adding those directories to the Sage config code where it is currently looking for stuff, is probably easier for a Sage developer unfamiliar with Macports, than for a person experienced with Macports that has no clue about Sage guts (and isn't eager to dive into those! ;).

the MacPorts-specific part would be to create a list of packages to install, and put it into build/pkgs/*/distros/macports 

and a similar global location.

Second, pplpy-0.8.4 is "obtuse" enough to try compiling C++ file linear_algebra.cpp with clang instead of clang++, and then complain that CFLAGS that are (surprise!) C-specific, do not make sense for C++. Need I mention that it is impossible to locate clang but not clang++?

pplpy upstream is here, it has a very busy maintainer who does not use macOS

https://gitlab.com/videlec/pplpy

- the main developer no longer works on SageMath, as far as I know.

patches are most welcome!

I'll take a look later, but the likelihood of me being able to fix it isn't great.

 

Platform

- iMac 2020, Intel Core i9 CPU, 32GM RAM

- macOS Catalina 10.15.7

- Xcode-12.3 (with command-line tools installed) in /Applications/Xcode.app, and separately installed CLT 12.3 in /Library/Developer/CommandLineTools.

XCode is a moving target, 9.2 was released before XCode 12.3

True, but not really relevant in the context. The big difference/jump came with Xcode-10. Xcode-11 and -12 exacerbated that a bit, but I daresay, imperceptibly so.

 

- Macports 2.6.4, with a ton of packages installed in /opt/local, which include clang-11, gcc-10, python38 and python39, and OpenSSL-1.1.1i.

one ought to use XCode's clang(++), and only need gfortran from the gcc package.

So one does. ;-)

As you can see from the log, the build uses Xcode's clang and clang++. 

For my stuff, especially when I need to deal with sanitizers extensively, I use Macports clang. And, of course, Xcode's GCC really sucks, so installing a "real" one from Macports was a-must.

 

Fortran binary location can be explicitly provided by setting FC environment variable.

(same for C and C++, one can use CC and CXX)

It's all done in my config - except for Fortran. I don't use it any more (for the last 20 years, or so ;), but it's no problem installing it. Assuming Sage would be smart enough to use it (rather than downloading and building it's own GCC and GFortran)!

 

If you like to enable MacPorts, think about providing an analog of .homebrew-build-env script.

That's something constructive that I can probably do. Thanks for bringing it up. 

--
You received this message because you are subscribed to the Google Groups "sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-support...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/717dd948-f1c2-4e18-9708-1dd2e68e277bn%40googlegroups.com.

[Mouse Mousevich]

We don't have a single MacPort user among a handful of SageMath developers. Please feel free to provide patches. (Yes, it means supporting yet another nonstandard location, more or less, I suppose)

The fun part is that to support Macports, you need to look in /opt/local/include for header files, in /opt/local/lib for libraries, and in /opt/local/bin for executables. How hard does that sound?

And I daresay, that adding those directories to the Sage config code where it is currently looking for stuff, is probably easier for a Sage developer unfamiliar with Macports, than for a person experienced with Macports that has no clue about Sage guts (and isn't eager to dive into those! ;).

the MacPorts-specific part would be to create a list of packages to install, and put it into build/pkgs/*/distros/macports 

and a similar global location.

I'm not sure I understand what that means. If a package is already installed by Macports - it does not make sense installing it again. More so, Macports keeps a system-wide installation, rather than a copy for each user.

It might make sense telling Macports to install packages that aren't already installed. But again, I don't see any benefit or even use in something like build/pkgs/*/distros/macports.

And, frankly, I see more harm than benefit in Sage getting stuck with copies of packages it builds itself in its private location, becoming bloated and stale at the same time. While the package manager (Macports in this case) keeps track of everything it "owns", shares one copy among all the users, and ensures they're updated.

[Mouse Mousevich]

I looked into build/pgks/* (1.5 GB) and it appears that Sage is trying to be a package manager, in addition to what people probably want it for. 

I guess at version 9.2 it's rather late to ask why it isn't good enough to just list in the README a set of packages that Sage requires to run, and have a minimal config that would check for their presence at install time? Letting user to worry about, e.g., where and how to install OpenSSL-1.1.x or python38?

[Samuel Lelièvre]

Le mer. 23 déc. 2020 à 14:55, Mouse Mousevich:

>>
>> the MacPorts-specific part would be to create a list of packages to install, and put it into build/pkgs/*/distros/macports
>> and a similar global location.
>
> I'm not sure I understand what that means. If a package is already installed
> by Macports - it does not make sense installing it again. More so, Macports
> keeps a system-wide installation, rather than a copy for each user.

For each package `pkgname` Sage uses, the `build/pkg/pkgname` folder
contains information about building and about not-building this package.
The not-building information is a folder `distros` with a bunch of text files,
one per distro, containing the name of the package in that distro that can
be used if present instead of building it. For an example, see

https://github.com/sagemath/sage/tree/develop/build/pkgs/openssl/distros

The various text files there tell the Sage build system what package
to check for. Depending on the distro, that might be one of:

libressl-openssl
libssl-dev
libssl-devel
openssl
openssl openssl-devel
openssl openssl-solibs
openssl-dev
security/openssl

To support using the packages from MacPorts, we need

- a file build/pkgs/macports.txt with information on core packages,
see all the text files in

https://github.com/sagemath/sage/tree/develop/build/pkgs/

- and for each "spkg", a macports.txt file containing the name of
the equivalent MacPorts package to check for (and to recommend
installing if not present)

See recent work on adding support for voidlinux

https://trac.sagemath.org/query?order=id&desc=1&summary=~voidlinux&summary=~void%20linux

[John H Palmieri]

The short answer is that Sage is designed for (among others) research mathematicians, who may not have the interest or inclination to learn how to install lots of system packages. So from the beginning it included as many components as possible. Years ago the presence of MacPorts and other similar systems would actually break the Sage build process, so the build process may attempt to disable them. Whether it tried to disable them, for a long time Sage refused to use any system packages and just built everything itself (with a few exceptions, like gcc).

This has been changing recently, and now Sage can use many system packages. This is restricted to systems (OS + package manager) used by Sage developers, and apparently no developer uses MacPorts, or at least not enough to spend time getting Sage to work with it. Homebrew is pretty well integrated with the Sage build process.

[Dima Pasechnik]

On Wed, 23 Dec 2020, 14:11 Mouse Mousevich, <u...@ll.mit.edu> wrote:

I looked into build/pgks/* (1.5 GB) and it appears that Sage is trying to be a package manager, in addition to what people probably want it for. 

I guess at version 9.2 it's rather late to ask why it isn't good enough to just list in the README a set of packages that Sage requires to run, and have a minimal config that would check for their presence at install time? Letting user to worry about, e.g., where and how to install OpenSSL-1.1.x or python38?

we are trying to do it right, with automated testing on the many platforms: a dozen of different Linux distributions,  some with several versions, with Conda, with Homebrew, with Cygwin and WSL on Windows.

This needs the information on what packages are needed on each of these.

Letting the user worry about won't help for this task.

--
You received this message because you are subscribed to the Google Groups "sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-support...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/85d43727-0336-406f-b877-35b5ed8bb29dn%40googlegroups.com.

[Matthias Koeppe]

On Wednesday, December 23, 2020 at 5:55:38 AM UTC-8 u...@ll.mit.edu wrote:

the MacPorts-specific part would be to create a list of packages to install, and put it into build/pkgs/*/distros/macports 

and a similar global location.

I'm not sure I understand what that means.

You can learn about it by reading https://doc.sagemath.org/html/en/developer/packaging.html#allowing-for-the-use-of-system-packages

If you want to help with supporting MacPorts packages, https://trac.sagemath.org/ticket/30504