3 ModuleNotFoundError with Ubuntu 18.04 + 9.2beta12 + #30053 + #30606

[Sébastien Labbé]

With Ubuntu 18.04 + 9.2beta12 + #30053 (so that it uses sage's python 3.8 instead of system 3.6 which can't get the doc to build) + #30606, I don't know if it is noise yet, but I obtain what's below.

Currently doing make distclean, etc. I am pasting it here instead of in #30606 to avoid noise on the ticket.

{{{
***************************************************************
Error building Sage.

The following package(s) may have failed to build (not necessarily
during this run of 'make all-start'):

* package:         conway_polynomials-0.5
  last build time: sept. 20 17:39
  log file:        /home/slabbe/GitBox/sage/logs/pkgs/conway_polynomials-0.5.log
  build directory: /home/slabbe/GitBox/sage/local/var/tmp/sage/build/conway_polynomials-0.5

* package:         setuptools-49.6.0
  last build time: sept. 20 17:39
  log file:        /home/slabbe/GitBox/sage/logs/pkgs/setuptools-49.6.0.log
  build directory: /home/slabbe/GitBox/sage/local/var/tmp/sage/build/setuptools-49.6.0

* package:         cryptominisat-5.6.8
  last build time: sept. 20 17:39
  log file:        /home/slabbe/GitBox/sage/logs/pkgs/cryptominisat-5.6.8.log                                     build directory: /home/slabbe/GitBox/sage/local/var/tmp/sage/build/cryptominisat-5.6.8
}}}

{{{
Thread model: posix
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)
****************************************************
Uninstalling existing 'setuptools'
Warning: File '/home/slabbe/GitBox/sage/local/lib/python3.6/site-packages/setuptools/command/launcher' not found
Warning: File '/home/slabbe/GitBox/sage/local/lib/python3.6/site-packages/setuptools/script' not found
Warning: File '/home/slabbe/GitBox/sage/local/(dev).tmpl' not found
Warning: File '/home/slabbe/GitBox/sage/local/manifest.xml' not found
Removing stamp file '/home/slabbe/GitBox/sage/local/var/lib/sage/installed/setuptools-49.6.0'
Traceback (most recent call last):
  File "setup.py", line 10, in <module>
    import setuptools
  File "/home/slabbe/GitBox/sage/local/var/tmp/sage/build/setuptools-49.6.0/src/setuptools/__init__.py", line 16, in <module>
    from setuptools.extern.six import PY3, string_types
  File "/home/slabbe/GitBox/sage/local/var/tmp/sage/build/setuptools-49.6.0/src/setuptools/extern/__init__.py", line 50, in load_module
    raise ImportError(
ImportError: The 'six' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.
******************************************************************************************
Error building / installing setuptools for Python 3
******************************************************************************************
}}}

{{{
Thread model: posix
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)
****************************************************
Warning: No files to uninstall for 'conway_polynomials'
Uninstalling existing 'conway_polynomials'
Removing stamp file '/home/slabbe/GitBox/sage/local/var/lib/sage/installed/conway_polynomials-0.5'
Traceback (most recent call last):
  File "spkg-install.py", line 2, in <module>
    import pickle
  File "/usr/lib/python3.8/pickle.py", line 33, in <module>
    from struct import pack, unpack
  File "/usr/lib/python3.8/struct.py", line 13, in <module>
    from _struct import *
ModuleNotFoundError: No module named '_struct'
real    0m0.023s
user    0m0.022s
sys    0m0.000s
************************************************************************
Error installing package conway_polynomials-0.5
************************************************************************
}}}

{{{
[ 97%] Building CXX object cmsat5-src/CMakeFiles/cryptominisat5_simple-bin.dir/main_simple.cpp.o
cd /home/slabbe/GitBox/sage/local/var/tmp/sage/build/cryptominisat-5.6.8/src/pycryptosat && /home/slabbe/GitBox/sage/local/bin/python3 setup.py build_ext --inplace --rpath ../lib
cd /home/slabbe/GitBox/sage/local/var/tmp/sage/build/cryptominisat-5.6.8/src/cmsat5-src && /home/slabbe/GitBox/sage/local/libexec/ccache/g++  -std=gnu++11  -DBOOST_TEST_DYN_LINK -DUSE_GAUSS -DUSE_M4RI -DUSE_ZLIB -I/home/slabbe/GitBox/sage/local/var/tmp/sage/build/cryptominisat-5.6.8/src -I/home/slabbe/GitBox/sage/local/var/tmp/sage/build/cryptominisat-5.6.8/src/cmsat5-src  -mtune=native -Wall -Wextra -Wunused -Wsign-compare -fno-omit-frame-pointer -Wtype-limits -Wuninitialized -Wno-deprecated -Wstrict-aliasing -Wpointer-arith -Wpointer-arith -Wformat-nonliteral -Winit-self -Wparentheses -Wunreachable-code -ggdb3 -Wlogical-op -Wrestrict -Wnull-dereference -Wdouble-promotion -Wshadow -Wformat=2 -pedantic   -g -pthread -O2 -fPIC -std=gnu++11 -o CMakeFiles/cryptominisat5_simple-bin.dir/main_simple.cpp.o -c /home/slabbe/GitBox/sage/local/var/tmp/sage/build/cryptominisat-5.6.8/src/src/main_simple.cpp
Traceback (most recent call last):
  File "/usr/lib/python3.8/subprocess.py", line 64, in <module>
    import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 111, in <module>
    if platform.system() == 'Darwin':
  File "/usr/lib/python3.8/platform.py", line 891, in system
    return uname().system
  File "/usr/lib/python3.8/platform.py", line 857, in uname
    processor = _syscmd_uname('-p', '')
  File "/usr/lib/python3.8/platform.py", line 611, in _syscmd_uname
    import subprocess
  File "/usr/lib/python3.8/subprocess.py", line 69, in <module>
    import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
pycryptosat/CMakeFiles/python_interface.dir/build.make:60: recipe for target 'pycryptosat/build/timestamp' failed
make[7]: *** [pycryptosat/build/timestamp] Error 1
CMakeFiles/Makefile2:283: recipe for target 'pycryptosat/CMakeFiles/python_interface.dir/all' failed
make[6]: *** [pycryptosat/CMakeFiles/python_interface.dir/all] Error 2
make[6]: *** Waiting for unfinished jobs....
[100%] Linking CXX executable ../cryptominisat5_simple
cd /home/slabbe/GitBox/sage/local/var/tmp/sage/build/cryptominisat-5.6.8/src/cmsat5-src && /usr/bin/cmake -E cmake_link_script CMakeFiles/cryptominisat5_simple-bin.dir/link.txt --verbose=ON
/home/slabbe/GitBox/sage/local/libexec/ccache/g++  -std=gnu++11   -mtune=native -Wall -Wextra -Wunused -Wsign-compare -fno-omit-frame-pointer -Wtype-limits -Wuninitialized -Wno-deprecated -Wstrict-aliasing -Wpointer-arith -Wpointer-arith -Wformat-nonliteral -Winit-self -Wparentheses -Wunreachable-code -ggdb3 -Wlogical-op -Wrestrict -Wnull-dereference -Wdouble-promotion -Wshadow -Wformat=2 -pedantic   -Wl,-rpath-link,/home/slabbe/GitBox/sage/local/lib -L/home/slabbe/GitBox/sage/local/lib -Wl,-rpath,/home/slabbe/GitBox/sage/local/lib -O2 -Wl,--discard-all -Wl,--build-id=sha1 -rdynamic CMakeFiles/cryptominisat5_simple-bin.dir/main_simple.cpp.o  -o ../cryptominisat5_simple -Wl,-rpath,/home/slabbe/GitBox/sage/local/var/tmp/sage/build/cryptominisat-5.6.8/src/lib: ../lib/libcryptominisat5.so.5.6 /usr/lib/x86_64-linux-gnu/libz.so /usr/lib/x86_64-linux-gnu/libm4ri.so -pthread
[100%] Built target cryptominisat5_simple-bin
Makefile:129: recipe for target 'all' failed
make[5]: *** [all] Error 2
******************************************************************************************
Error building cryptominisat-5.6.8
******************************************************************************************
}}}

[Sébastien Labbé]

make-distclean and make fixed it.

It was related to building python3.8 without first doing make python3-clean

Sorry for the noise.

[Dima Pasechnik]

by the way, Ubuntu 18.04 has a Python 3.8 package, so you can use it
by running configure with including
PYTHON3=/usr/bin/python3.8
in its arguments, e.g

./configure PYTHON3=/usr/bin/python3.8 # and maybe more arguments

(do not forget "make python3-clean" before this)

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/d1b2050f-14ff-4135-8a32-726194a78fb2n%40googlegroups.com.

[Sébastien Labbé]

I did installed python3.8 on the system, but sage does not pick it because is does not have distutils if I remember correctly (config.log was overwritten since then and now says "configure:29566: python3 has already been installed by SageMath")

How should I install a python3.8 on Ubuntu18.04 that satisfies all the requirements?

[Dima Pasechnik]

On Mon, Sep 21, 2020 at 10:36 AM Sébastien Labbé <sla...@gmail.com> wrote:
>
> I did installed python3.8 on the system, but sage does not pick it because is does not have distutils if I remember correctly (config.log was overwritten since then and now says "configure:29566: python3 has already been installed by SageMath")

how come ? seems to be in order?

Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-112-generic i686)
...
dima@arando:~$ python3.8
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import distutils
>>>
>>>
dima@arando:~$ uname -a
Linux arando 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:06 UTC
2020 i686 i686 i686 GNU/Linux

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/3033fa55-9a23-457f-bcc4-7b15c18f905fn%40googlegroups.com.

[Sébastien Labbé]

Starting from scratch, here is what configure says about my system Python 3.8:

...

-----------------------------------------------------------------------------

Checking whether SageMath should install SPKG python3...
checking whether any of sqlite libpng bzip2 xz libffi is installed as or will be installed as SPKG... no
checking for python3 >= 3.7, < 3.9 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core...
checking ... whether /usr/bin/python3.8 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension
checking ... whether /usr/bin/python3 is good... no, Python 3.6.9 is too old
configure: no suitable system package found for SPKG python3
-----------------------------------------------------------------------------

...

It does recommend me this:

configure: hint: installing the following system packages is recommended and may avoid building some of the above SPKGs from source:
configure:   $ sudo apt-get update
  $ sudo apt-get install libboost-dev pari-gp2c libisl-dev libnauty-dev python3 libpython3-dev python3-distutils

But these are already installed. In particular:

$ sudo apt-get install python3-distutils
Lecture des listes de paquets... Fait
Construction de l'arbre des dépendances       
Lecture des informations d'état... Fait
python3-distutils est déjà la version la plus récente (3.6.9-1~18.04).
0 mis à jour, 0 nouvellement installés, 0 à enlever et 0 non mis à jour.

seems related to python3.6...

[Dima Pasechnik]

On Mon, Sep 21, 2020 at 11:24 AM Sébastien Labbé <sla...@gmail.com> wrote:
>
> Starting from scratch, here is what configure says about my system Python 3.8:
>
> ...
> -----------------------------------------------------------------------------
> Checking whether SageMath should install SPKG python3...
> checking whether any of sqlite libpng bzip2 xz libffi is installed as or will be installed as SPKG... no
> checking for python3 >= 3.7, < 3.9 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core...
> checking ... whether /usr/bin/python3.8 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension
> checking ... whether /usr/bin/python3 is good... no, Python 3.6.9 is too old

try
https://trac.sagemath.org/ticket/30053
which bans 3.6.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/41c1a42c-6ccf-4cd2-bd4b-a410fa5cb890n%40googlegroups.com.

[Dima Pasechnik]

On Mon, Sep 21, 2020 at 11:52 AM Dima Pasechnik <dim...@gmail.com> wrote:
>
> On Mon, Sep 21, 2020 at 11:24 AM Sébastien Labbé <sla...@gmail.com> wrote:
> >
> > Starting from scratch, here is what configure says about my system Python 3.8:
> >
> > ...
> > -----------------------------------------------------------------------------
> > Checking whether SageMath should install SPKG python3...
> > checking whether any of sqlite libpng bzip2 xz libffi is installed as or will be installed as SPKG... no
> > checking for python3 >= 3.7, < 3.9 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core...
> > checking ... whether /usr/bin/python3.8 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension
> > checking ... whether /usr/bin/python3 is good... no, Python 3.6.9 is too old
>
> try
> https://trac.sagemath.org/ticket/30053
> which bans 3.6.

you might need to

apt install python3-distutils

and then

./configure PYTHON3=/usr/bin/python3.8

[Sébastien Labbé]

> > -----------------------------------------------------------------------------
> > Checking whether SageMath should install SPKG python3...
> > checking whether any of sqlite libpng bzip2 xz libffi is installed as or will be installed as SPKG... no
> > checking for python3 >= 3.7, < 3.9 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core...
> > checking ... whether /usr/bin/python3.8 is good... no, the version is in the supported range, and the modules can be imported, but distutils cannot build a C extension
> > checking ... whether /usr/bin/python3 is good... no, Python 3.6.9 is too old
>
> try
> https://trac.sagemath.org/ticket/30053
> which bans 3.6.

Actually, the above is obtained with the branch #30053. Otherwise, system python 3.6 is chosen.

 

you might need to

apt install python3-distutils

As I wrote, this is was I did. It says it is already the most recent version (3.6.9-1~18.04) which is installed, which seems to mean that it is distutils for system python 3.6 not 3.8.

 

and then

./configure PYTHON3=/usr/bin/python3.8

I can try that, but I think it will not work since "distutils cannot build a C extension" in my system python 3.8.

[Matthias Koeppe]

Try installing python3.8-distutils. 

[Dima Pasechnik]

On Mon, 21 Sep 2020, 16:42 Matthias Koeppe, <matthia...@gmail.com> wrote:

Try installing python3.8-distutils. 

indeed. it confusingly installs

python3-distutils instead, but the latter has distutils for all python3.x for 5<x<9

this works for me.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/3534f6be-b7e2-422a-9cf2-d1cc59c8d1ccn%40googlegroups.com.

[Sébastien Labbé]

Got it!

On Monday, September 21, 2020 at 5:42:28 PM UTC+2 Matthias Koeppe wrote:

Try installing python3.8-distutils. 

this package does not exist, trying to sudo apt-get install python3.8-distutils replaces it by python3-distutils which is already installed.

What worked for me (on ubuntu 18.04 I recall) is:

sudo apt-get install python3.8-dev

and now ./configure picks up the system Python:

-----------------------------------------------------------------------------
Checking whether SageMath should install SPKG python3...
checking whether any of sqlite libpng bzip2 xz libffi is installed as or will be installed as SPKG... no
checking for python3 >= 3.7, < 3.9 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core...

checking ... whether /usr/bin/python3.8 is good... yes
checking for python3 >= 3.6, < 3.9 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core... /usr/bin/python3.8
configure: will use system package and not install SPKG python3
-----------------------------------------------------------------------------