pointy python question related to sage

[François Bissey]

Hi all,

I was wondering if there was a way to figure if a module from one
package is imported from another one and which one at that.

This is quite relevant to sage-on-distros. mpmath-1.2.0+ tries
to figure out if it is called from sage by detecting if SAGE_ROOT
is defined. Assuming that if it is this is a sage install.
Some reasoning at
https://github.com/fredrik-johansson/mpmath/pull/466
the reasoning in question disregarded the fact that a sage-on-distro
package would also likely use a system mpmath.

But mpmath developers don’t want to just use sage if it is installed.
Only if it is explicitly required and available or because they
mpmath is called from sage.

MPMATH_SAGE could be set in the sage executable or sage-env but
that would cover the case where you import sage directly from
python. Something that has been doable on sage-on-(some)-distros
for years and is quite doable in villa sage these days.

So I am wondering if there is a python way to say to mpmath, you
have been imported from sage.

Cheers,
François

[Nils Bruin]

Am I understanding correctly that "import mpmath" should behave differently when called in a sage context rather than in a straight python context, and that (in case of a system python) it may be possible that within the same python installation both behaviours are desirable on occasion?

From what I understand, python packages upon import may sometime probe availability of certain other packages (by trying to import them) and may fall back on other options if they are not. This allows certain "optional dependencies".

In this case, it doesn't sound like an optional dependencies, but really as two different modes for mpmath. It seems to me one should select between those modes explicitly; whether through an environment variable or some flag inside python.

Design-wise this causes another problem: if a python package relies on mpmath in non-sage mode, then you can now not use it inside sage. That's an unfortunate incompatibility and something one might generally want to avoid.

[François Bissey]

You have the gist of it. There are various environment variables that can suppress or enable
the use of sage as a back for multi-precision numbers. Historically that sometime caused trouble
in sympy as well.

> On 19/03/2021, at 05:28, Nils Bruin <nbr...@sfu.ca> wrote:
>
> Am I understanding correctly that "import mpmath" should behave differently when called in a sage context rather than in a straight python context, and that (in case of a system python) it may be possible that within the same python installation both behaviours are desirable on occasion?
>
> From what I understand, python packages upon import may sometime probe availability of certain other packages (by trying to import them) and may fall back on other options if they are not. This allows certain "optional dependencies".
>
> In this case, it doesn't sound like an optional dependencies, but really as two different modes for mpmath. It seems to me one should select between those modes explicitly; whether through an environment variable or some flag inside python.
>

Indeed and from the mpmath dev point of view, they want the default mode to
be without sage, unless explicitly enabled. But at the moment the only way forward
I have as a sage-on-gentoo packager is to reverse those default once sage is installed.

If you are only ever using sage from the sage executable, you could set it in sage-env
so it would be set when you start sage. But in sage-on-distro and increasingly in mainstream
sage, a good amount of work has been put in the design of env.py so that you can import sage
from python directly. In which case you bypass the sage executable.
Apart from installation (work in progress), sage is pretty much a regular python package these days.

> Design-wise this causes another problem: if a python package relies on mpmath in non-sage mode, then you can now not use it inside sage. That's an unfortunate incompatibility and something one might generally want to avoid.

While there could be numerical noise between backends, I would say that
this kind of problems means you are using something that shouldn’t be
exposed to the user, a private API.

This is probably something bad sage does as well. When mpmath is in non
sage mode and called from sage you get all sorts of failures. For example:

sage -t --long --random-seed=0 /usr/lib/python3.9/site-packages/sage/calculus/calculus.py
**********************************************************************
File "/usr/lib/python3.9/site-packages/sage/calculus/calculus.py", line 372, in sage.calculus.calculus
Failed example:
[f[0].n() for f in _.coefficients()] # numerical coefficients to make comparison easier; Maple 12 gives same answer
Exception raised:
Traceback (most recent call last):
File "sage/symbolic/expression.pyx", line 6186, in sage.symbolic.expression.Expression.numerical_approx (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/symbolic/expression.cpp:37043)
x = x._convert(kwds)
File "sage/symbolic/expression.pyx", line 1495, in sage.symbolic.expression.Expression._convert (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/symbolic/expression.cpp:10451)
cdef GEx res = self._gobj.evalf(0, kwds)
File "sage/libs/pynac/pynac.pyx", line 2139, in sage.libs.pynac.pynac.py_psi2 (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/pynac/pynac.cpp:25306)
return mpmath_utils.call(mpmath.psi, n, x, prec=prec)
File "sage/libs/mpmath/utils.pyx", line 439, in sage.libs.mpmath.utils.call (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/mpmath/utils.c:7129)
y = mpmath_to_sage(y, prec)
File "sage/libs/mpmath/utils.pyx", line 272, in sage.libs.mpmath.utils.mpmath_to_sage (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/mpmath/utils.c:5518)
mpfr_from_mpfval(y.value, x._mpf_)
File "sage/libs/mpmath/utils.pyx", line 167, in sage.libs.mpmath.utils.mpfr_from_mpfval (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/mpmath/utils.c:4702)
sign, man, exp, bc = x
TypeError: Cannot convert mpz to sage.rings.integer.Integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/sage/doctest/forker.py", line 714, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/usr/lib/python3.9/site-packages/sage/doctest/forker.py", line 1133, in compile_and_execute
exec(compiled, globs)
File "<doctest sage.calculus.calculus[101]>", line 1, in <module>
[f[Integer(0)].n() for f in _.coefficients()] # numerical coefficients to make comparison easier; Maple 12 gives same answer
File "<doctest sage.calculus.calculus[101]>", line 1, in <listcomp>
[f[Integer(0)].n() for f in _.coefficients()] # numerical coefficients to make comparison easier; Maple 12 gives same answer
File "sage/structure/element.pyx", line 885, in sage.structure.element.Element.n (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/structure/element.c:8385)
return self.numerical_approx(prec, digits, algorithm)
File "sage/symbolic/expression.pyx", line 6190, in sage.symbolic.expression.Expression.numerical_approx (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/symbolic/expression.cpp:37116)
x = x._convert(kwds)
File "sage/symbolic/expression.pyx", line 1495, in sage.symbolic.expression.Expression._convert (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/symbolic/expression.cpp:10451)
cdef GEx res = self._gobj.evalf(0, kwds)
File "sage/libs/pynac/pynac.pyx", line 2139, in sage.libs.pynac.pynac.py_psi2 (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/pynac/pynac.cpp:25306)
return mpmath_utils.call(mpmath.psi, n, x, prec=prec)
File "sage/libs/mpmath/utils.pyx", line 439, in sage.libs.mpmath.utils.call (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/mpmath/utils.c:7129)
y = mpmath_to_sage(y, prec)
File "sage/libs/mpmath/utils.pyx", line 272, in sage.libs.mpmath.utils.mpmath_to_sage (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/mpmath/utils.c:5518)
mpfr_from_mpfval(y.value, x._mpf_)
File "sage/libs/mpmath/utils.pyx", line 167, in sage.libs.mpmath.utils.mpfr_from_mpfval (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/mpmath/utils.c:4702)
sign, man, exp, bc = x
TypeError: Cannot convert mpz to sage.rings.integer.Integer
**********************************************************************
1 item had failures:
1 of 113 in sage.calculus.calculus
[429 tests, 1 failure, 9.93 s]
----------------------------------------------------------------------
sage -t --long --random-seed=0 /usr/lib/python3.9/site-packages/sage/calculus/calculus.py # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 10.0 seconds

More of those at https://github.com/cschwan/sage-on-gentoo/issues/628

François

[Samuel Lelievre]

See also

https://trac.sagemath.org/ticket/25445