Issue with external packages using Cython

[Marc Mezzarobba]

Hi everyone,

Some external packages that cimport cython modules from sage no
longer build with sage 9.7.beta8. It looks like this is due to
some recent change in Sage, as is reportedly works with sage 9.6.

Two examples:

~/co/sage:develop$ ./sage -pip install sage-flatsurf
Collecting sage-flatsurf
Using cached sage_flatsurf-0.4.7-py3-none-any.whl (206 kB)
Collecting surface-dynamics
Using cached surface_dynamics-0.4.7.tar.gz (8.3 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [337 lines of output]
/home/marc/co/sage/local/var/lib/sage/venv-python3.10/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-2fum1bmr/surface-dynamics_04b48bb6529e4830942d5ec0e4b492bb/surface_dynamics/flat_surfaces/origamis/origami_dense.pxd
tree = Parsing.p_module(s, pxd, full_module_name)

Error compiling Cython file:
------------------------------------------------------------
...
# Otherwise we end up with the strange error
# sage: from .origami import Origami
# Traceback (most recent call last):
# ...
# AttributeError: 'module' object has no attribute 'ModuleElementWithMutability'
cimport sage.structure.element
^
------------------------------------------------------------

surface_dynamics/flat_surfaces/origamis/origami_dense.pyx:39:8: 'sage/structure/element.pxd' not found
[...]
File "/home/marc/co/sage/local/var/lib/sage/venv-python3.10/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1250, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: surface_dynamics/flat_surfaces/origamis/origami_dense.pyx
Adding extension surface_dynamics.flat_surfaces.origamis.origami_dense:
sources = ['origami_dense.pyx', 'normal_form.c', 'lyapunov_exponents.c']
headers = ['origami_dense.pxd', 'lyapunov_exponents.h', 'normal_form.h']
Adding extension surface_dynamics.interval_exchanges.lyapunov_exponents:
sources = ['lyapunov_exponents.pyx', 'generalized_permutation.c', 'lin_alg.c', 'quad_cover.c', 'random.c', 'permutation.c']
headers = ['lyapunov_exponents.h']
Adding extension surface_dynamics.interval_exchanges.integer_iet:
sources = ['integer_iet.pyx', 'int_iet.c', 'int_vector.c']
headers = ['integer_iet.pxd', 'int_iet.h']
Adding extension surface_dynamics.interval_exchanges.iet_family:
sources = ['iet_family.pyx']
headers = []
Compiling surface_dynamics/flat_surfaces/origamis/origami_dense.pyx because it changed.
Compiling surface_dynamics/interval_exchanges/lyapunov_exponents.pyx because it changed.
Compiling surface_dynamics/interval_exchanges/integer_iet.pyx because it changed.
Compiling surface_dynamics/interval_exchanges/iet_family.pyx because it depends on /home/marc/co/sage/local/var/lib/sage/venv-python3.10/lib/python3.10/site-packages/ppl/ppl_decl.pxd.
[1/4] Cythonizing surface_dynamics/flat_surfaces/origamis/origami_dense.pyx
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
(exit 1)

or:

~/co/sage:develop$ ./sage -pip install git+https://github.com/mkauers/ore_algebra.git
Collecting git+https://github.com/mkauers/ore_algebra.git
Cloning https://github.com/mkauers/ore_algebra.git to /tmp/pip-req-build-r3prw8dd
Running command git clone --filter=blob:none --quiet https://github.com/mkauers/ore_algebra.git /tmp/pip-req-build-r3prw8dd
Resolved https://github.com/mkauers/ore_algebra.git to commit 47e05a4556c854847f0ed9239fc3e288fde28ab3
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [1582 lines of output]

Error compiling Cython file:
------------------------------------------------------------
...
# Distributed under the terms of the GNU General Public License (GPL) either
# version 2, or (at your option) any later version
#
# http://www.gnu.org/licenses/

from sage.libs.arb.types cimport *
^
------------------------------------------------------------

src/ore_algebra/analytic/binary_splitting_arb.pyx:16:0: 'sage/libs/arb/types.pxd' not found
[...]
Cython.Compiler.Errors.CompileError: src/ore_algebra/analytic/binary_splitting_arb.pyx
Compiling src/ore_algebra/analytic/binary_splitting_arb.pyx because it changed.
Compiling src/ore_algebra/analytic/eval_poly_at_int.pyx because it changed.
Compiling src/ore_algebra/analytic/naive_sum_c.pyx because it changed.
[1/3] Cythonizing src/ore_algebra/analytic/binary_splitting_arb.pyx
[end of output]
[...]

Does anyone know what the issue might be?

Thanks,

--
Marc

[Matthias Koeppe]

See https://trac.sagemath.org/wiki/ReleaseTours/sage-9.7#sagesage.ringssage.libs...arenowPEP420namespacepackages

and https://trac.sagemath.org/ticket/34221

[Marc Mezzarobba]

Matthias Koeppe wrote:
> See
>
https://trac.sagemath.org/wiki/ReleaseTours/sage-9.7#sagesage.ringssage.libs...arenowPEP420namespacepackages
> and https://trac.sagemath.org/ticket/34221

Thanks a lot!

--
Marc