Python version support for SymPy/Sage

[Oscar Benjamin]

Hi all,

My question here is: would it be problematic for Sage if SymPy were to
follow SPEC 0/NEP 29 which would mean dropping support for older
Python versions more quickly?

We are about to release SymPy 1.13 which will support Python 3.8 to
3.13. It has been asked on the SymPy mailing list whether SymPy could
adopt SPEC 0 which is basically the same as NEP 29:

https://groups.google.com/g/sympy/c/VNz2xJ1Sywo/m/0H1-KmaJAgAJ?utm_medium=email&utm_source=footer

https://scientific-python.org/specs/spec-0000/

There are older discussions about this as well:

https://github.com/sympy/sympy/issues/21884

This relates to previous discussions on this list about Sage following NEP 29:

https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/Ap_1pHlsBQAJ?utm_medium=email&utm_source=footer
https://groups.google.com/g/sage-devel/c/3Zoq0CNE1hE/m/m40v2e_cBwAJ

Ordinarily SymPy would have dropped support for Python 3.8 by now
anyway regardless of SPEC 0 or NEP 29. I can't remember where this was
discussed but I think that the reason Python 3.8 is still supported is
just because we thought it was needed by Sage.

Following those specifications in coordination with other scientific
Python packages would mean dropping both 3.8 and 3.9 now and then also
dropping 3.10 in a few months. I don't propose to do this right now
with the SymPy 1.13 release but once 1.13 is released I want to drop
at least 3.8 if not also 3.9 and then ideally we would have a clear
policy for this going forwards.

It is not particularly difficult for SymPy itself to support a wide
range of Python versions but it creates potential problems for other
packages that depend on SymPy. For example when SymPy 1.13 is released
(soon) any packages that depend on SymPy potentially need to put out
updates for all of the same Python versions that the SymPy release
claims support for. If the SymPy 1.13 update breaks something then
pushing it out to as many Python versions as possible is like
spreading the risk of breakage as far as possible. We need to be
careful about this because besides direct end users or Sage there are
also very widely used packages like pytorch that depend on SymPy which
has forced us into rushing out fix releases recently:
https://github.com/sympy/sympy/issues/26273

Personally I am in favour of SymPy coordinating with the rest of the
scientific Python ecosystem by adopting SPEC 0/NEP 29 so that there is
a clearly defined policy that is understood and depended on by
everything downstream. Old versions of SymPy will continue to work
with old versions of Python so people who like packaging old versions
of everything (e.g. Linux distros) can do so.

People who want to use the latest versions of Python packages should
also use recent versions of Python. My recommendation to a user who
just wants to use Python would generally be to install the second most
recent version of Python i.e. currently Python 3.11 but soon 3.12. I
would not recommend anyone to install Python 3.8 right now.

For now I use pyenv to install different Python versions but hopefully
in future it will become easier to install a particular Python version
from binary on any OS. I expect that soon uv will be able to do this
using the PyBI binaries at which point I will probably switch to using
uv in replacement of both pyenv and pip. I expect that in future
editors like vscode will be able to handle installing a particular
Python version and all the packages automatically for users who will
not even need to run uv/pip etc directly.

Conda already makes it easy to install particular Python versions for
people using conda. Windows/MacOS users will typically download the
newest Python version from the python.org download page. Linux distros
usually have ways of installing newer Python versions from repos and
it's not hard to build Python on Linux anyway.

It is not clear to me what the benefit is of supporting old Python
versions especially for Sage: building/installing Sage is a vastly
bigger issue than building/installing a new Python version.

My question then is would SymPy adopting SPEC 0/NEP 29 be a problem for Sage?

If SymPy 1.14 is released later this year and drops support for Python
3.8, 3.9 and 3.10 would that be a problem?

Oscar

[Matthias Koeppe]

On Wednesday, June 5, 2024 at 5:31:30 AM UTC-7 Oscar Benjamin wrote:

My question here is: would it be problematic for Sage if SymPy were to
follow SPEC 0/NEP 29 which would mean dropping support for older
Python versions more quickly?

Quick answer: Probably it would not be very problematic on our side because many of other upstream packages of Sage (NumPy, SciPy, NetworkX, ...) already follow that schedule. But it could mean that we sometimes hold off on updating SymPy to the latest version for a little while, like we currently do with other packages.

Example: SPEC 0/NEP 29 dropped Python 3.9 earlier this year, and NetworkX has already pushed out a version 3.3 which no longer supports Python 3.9.

But that's an outlier from our perspective; the released versions of NumPy and SciPy (including the upcoming NumPy 2.0) still support Python 3.9.

So without major cost to us, we are able to keep Python 3.9 a bit longer.

 

We are about to release SymPy 1.13 which will support Python 3.8 to
3.13.

Great, I hope we can still squeeze it in to our upcoming Sage 10.4 release!

 

It has been asked on the SymPy mailing list whether SymPy could
adopt SPEC 0 which is basically the same as NEP 29:

https://groups.google.com/g/sympy/c/VNz2xJ1Sywo/m/0H1-KmaJAgAJ?utm_medium=email&utm_source=footer
https://scientific-python.org/specs/spec-0000/

There are older discussions about this as well:

https://github.com/sympy/sympy/issues/21884

This relates to previous discussions on this list about Sage following NEP 29:

https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/Ap_1pHlsBQAJ?utm_medium=email&utm_source=footer
https://groups.google.com/g/sage-devel/c/3Zoq0CNE1hE/m/m40v2e_cBwAJ

We also have a wiki page about NEP 29 with many collected facts and discussion points:

https://github.com/sagemath/sage/wiki/NEP-29:-Python-version-strategy

(last edited by me in October 2023; should update for SPEC 0 etc.)

Ordinarily SymPy would have dropped support for Python 3.8 by now
anyway regardless of SPEC 0 or NEP 29. I can't remember where this was
discussed but I think that the reason Python 3.8 is still supported is
just because we thought it was needed by Sage.

We dropped Python 3.8 in the Sage 10.2 cycle (August 2023).

 

Following those specifications in coordination with other scientific
Python packages would mean dropping both 3.8 and 3.9 now and then also
dropping 3.10 in a few months. I don't propose to do this right now
with the SymPy 1.13 release but once 1.13 is released I want to drop
at least 3.8 if not also 3.9 

No problem; by the time that SymPy 1.14 is ready, we'll likely have dropped 3.9.

 

It is not particularly difficult for SymPy itself to support a wide
range of Python versions

Likewise for the Sage library. 

Personally I am in favour of SymPy coordinating with the rest of the
scientific Python ecosystem by adopting SPEC 0/NEP 29 so that there is
a clearly defined policy that is understood and depended on by
everything downstream. Old versions of SymPy will continue to work
with old versions of Python so people who like packaging old versions
of everything (e.g. Linux distros) can do so.

Yes, this makes sense and is, of course, very much the point of SPEC 0/NEP 29.

As Sage currently is "downstream of everything" and "upstream of nothing", we have effectively been following NEP 29 + an additive constant. See https://github.com/sagemath/sage/wiki/NEP-29:-Python-version-strategy#schedule-with-comparison-to-nep-29

(I hope that this will change in the longer term with the modularization of the Sage library, but currently it is not a relevant factor.)

[...Remarks on pyenv, uv, vscode, conda elided...]

It is not clear to me what the benefit is of supporting old Python
versions especially for Sage: building/installing Sage is a vastly
bigger issue than building/installing a new Python version.

One important factor regarding on our side has really been cultural rather than technical: To counter the perception that "Sage is so isolated from the rest of the world, it even brings its own Python", we have gone to great lengths to support the use of (venvs over) the system-provided Python; even though that goes counter to the common mantra among Python devs that "system Python is not for you". Because of this, the available (and default) Python versions on common distributions have been a factor in our version support strategy.

[Oscar Benjamin]

Thanks Matthias.

On Wed, 5 Jun 2024 at 19:32, Matthias Koeppe <matthia...@gmail.com> wrote:
> On Wednesday, June 5, 2024 at 5:31:30 AM UTC-7 Oscar Benjamin wrote:
>
> > Ordinarily SymPy would have dropped support for Python 3.8 by now
> > anyway regardless of SPEC 0 or NEP 29. I can't remember where this was
> > discussed but I think that the reason Python 3.8 is still supported is
> > just because we thought it was needed by Sage.
>
> We dropped Python 3.8 in the Sage 10.2 cycle (August 2023).

Okay, well we'll keep it in SymPy 1.13 for now. The release has
already been delayed long enough so I don't want to change anything at
this point.

> > Following those specifications in coordination with other scientific
> > Python packages would mean dropping both 3.8 and 3.9 now and then also
> > dropping 3.10 in a few months. I don't propose to do this right now
> > with the SymPy 1.13 release but once 1.13 is released I want to drop
> > at least 3.8 if not also 3.9
>
> No problem; by the time that SymPy 1.14 is ready, we'll likely have dropped 3.9.

Okay great. We'll drop both Python 3.8 and 3.9 then after sympy 1.13
is released.

You haven't mentioned 3.10. I infer from this that it currently seems
problematic for Sage to drop support for 3.10 in a few months (and
that it might be awkward if SymPy did so).

--
Oscar

[Matthias Koeppe]

Sorry for missing your question regarding 3.10.

That's really where it becomes relevant what "following the SPEC 0/NEP 29 schedule" really means for a project. 

(I have a bit of analysis on the actual practices of projects in our wiki page; it varies quite a bit, and more variation comes in simply from the coarse granularity of release schedules.)

If you drop Python 3.10 in preparation for a SymPy 1.14 release at the end of 2024, I think that you are likely going to be early in the pack with this, and in Sage we might be holding off on the update for a while -- unless there's a SymPy feature that is of particular importance for us, or we notice that Python 3.10 does not matter regarding system Python support on distributions, or NumPy/SciPy releases that drop the support are ready also, etc.

Matthias