Releasing SymPy 1.14

[Oscar Benjamin]

Hi all,

I think it is a good time to release SymPy 1.14. I was too busy in the
last few months to do this.

The main thing that needs to be decided before releasing is just what
the minimum supported Python version should be.

I bumped the minimum version from Python 3.8 to 3.9 because 3.8 is old
and was causing problems for this PR:
https://github.com/sympy/sympy/pull/27850

There is spec 0 which attempts to coordinate Python and other version
support in the scientific Python ecosystem:
https://scientific-python.org/specs/spec-0000/

Previous discussions about this are at:
https://groups.google.com/g/sympy/c/VNz2xJ1Sywo/m/0H1-KmaJAgAJ

In anticipation of the suggestion to write a policy/SymPEP about spec
0 or version support let me just point out that doing that before
releasing SymPy 1.14 will delay the release which is already overdue.
Right now we just need a quick decision about which versions SymPy
1.14 should support.

Following spec 0 would mean dropping support for 3.9 and 3.10 right
now meaning that the currently supported versions would be 3.11, 3.12
and 3.13 (and 3.14). I don't know of any strong advantage that SymPy
would get from dropping 3.9 or 3.10 in particular but there is a
reason why spec 0 says to do this regardless.

The problem with SymPy putting out a new release that would be
installed on older Python versions is that it can break other packages
that depend on SymPy but that no longer provide releases for those
older Python versions. Most of the time it is correct that Python
packages don't use upper version constraints like sympy<=1.13 but it
makes it problematic if SymPy and those other packages don't support
the same range of Python versions. This is basically why spec 0
recommends that projects in the scientific Python ecosystem coordinate
to support a common set of versions. Essentially part of the
compatibility information between different packages is encoded by
packages following a similar policy for Python version support.

On the other hand spec 0 is not universally adopted and takes a
somewhat aggressive approach to minimum supported Python versions.
CPython itself says that versions 3.9 to 3.13 are still considered
supported:
https://devguide.python.org/versions/

Two major dependents for SymPy are pytorch and SageMath. Currently
SageMath claims to support 3.9 to 3.12 although I think a new release
is coming soon that would support 3.13 and would perhaps drop some of
the older versions:
https://doc.sagemath.org/html/en/reference/spkg/python3.html

The last release of pytorch was a couple of months ago and supports 3.9 to 3.13:
https://pypi.org/project/torch/2.6.0/#files

Potentially if SymPy 1.14 does not support the whole range of Python
versions that e.g. torch does then that creates a problem for them
because it means they have to straddle multiple SymPy versions.
Ideally we don't break them in that situation but I think that every
SymPy release does cause issues for both torch and Sage. Currently
torch constrains sympy>=1.13.3 i.e. they only accept the absolutely
newest release of SymPy. I think Sage constrains the sympy version in
a similar way so it is useful for them if a SymPy release supports at
least the Python versions that they want to support.

For SymPy's own dependencies, mpmath's last release 1.3.0 was 2 years
ago and does not list any particular Python version support. The
current mpmath master branch has requires-python>=3.9 but I have no
idea when there might be a full release of mpmath.

Both gmpy2 and python-flint are optional dependencies. The last
release of gmpy2 was almost a year ago and provides binaries for
CPython 3.7 to 3.13. The latest release of python-flint only supports
3.11, 3.12 and 3.13.

Other things like numpy, scipy, matplotlib etc generally follow spec 0
so they currently only support 3.11-3.13.

I don't have strong opinions about Python version support for SymPy
3.14 but it needs to be decided before we can put out a new release.

--
Oscar

[Oscar Gustafsson]

Current Matplotlib (3.10) supports Python 3.10, but the next release, still some months away, will only claim support for 3.11+.

I'd say 3.9 (since that is working now) and then bump one, possibly two, version(s) after the release.

BR Oscar

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sympy/CAHVvXxQKm%3DVx3U53GN_GhRbAm-68c982NZtA1wfrk7G%2BAqEBww%40mail.gmail.com.

[Oscar Benjamin]

Thanks Oscar. Sticking with 3.9 for now does seem reasonable. I am
inclined to go with that if no one objects.

Having just fiddled around with the release notes pages I am reminded
that there is a Python version support policy:
https://github.com/sympy/sympy/wiki/Python-version-support-policy

The policy says that SymPy supports Python versions until they reach
EOL which would mean keeping 3.9 to 3.13 right now. I don't remember
exactly but I suspect that the original motivation for writing that
policy was really just to justify dropping Python 2.7 though.

--
Oscar

> To view this discussion visit https://groups.google.com/d/msgid/sympy/CAFjzj-J1D89xr%2Bqb8QpFqvLhZF9DcFQTkNohb4JWOKFCPss%2BvA%40mail.gmail.com.

[Amir Ebrahimi]

At our company, we'll be on Python 3.9 until EOL hits in October, so it would preferred that sympy continues to maintain that support until then.

[Jason Moore]

Dear Oscar,

I think we should support Python versions that are not yet EOL at our release time unless we find that it is particularly difficult to support an older Python version, which could be decided on a case-by-case basis. This helps maximize users ability to use the latest SymPy version. So I vote Python 3.9+ for this release.

Jason

moorepants.info
+01 530-601-9791

[Aaron Meurer]

That's been our policy in the past
https://github.com/sympy/sympy/wiki/Python-version-support-policy. We
definitely should just decide on a policy and stick with it. Then we
won't have to discuss it every time we do a release. The main downside
of supporting Python versions up to EOL is that Python has a faster
release cadence now, meaning that when we do this we have to support
up to 5 different Python versions at once (3.9, 3.10, 3.11, 3.12, and
3.13). When that policy was written the cadence was slower and only
meant supporting 3 versions at once (not counting Python 2).

At the same time, I'm not aware of new Python features that we really
wish we could be using but can't as long as we keep supporting older
versions. Are there examples of this?

Aaron Meurer

> To view this discussion visit https://groups.google.com/d/msgid/sympy/CAP7f1Ai1MvXUV6a0_OJsufda1a8o%2Bw%2BbzKbqp1XOdRNMzzPsuA%40mail.gmail.com.

[Oscar Benjamin]

On Tue, 15 Apr 2025 at 18:40, Aaron Meurer <asme...@gmail.com> wrote:
>
> That's been our policy in the past
> https://github.com/sympy/sympy/wiki/Python-version-support-policy. We
> definitely should just decide on a policy and stick with it. Then we
> won't have to discuss it every time we do a release. The main downside
> of supporting Python versions up to EOL is that Python has a faster
> release cadence now, meaning that when we do this we have to support
> up to 5 different Python versions at once (3.9, 3.10, 3.11, 3.12, and
> 3.13). When that policy was written the cadence was slower and only
> meant supporting 3 versions at once (not counting Python 2).

I think this is why spec 0 says what it does. It was decided to
support 3 Python versions when Python versions were released every 18
months which meant supporting all non-EOL Pythons. Then CPython
decided to go to a 12 month release schedule and moved to supporting 5
versions concurrently rather than 3.

Supporting many versions like this is not such a problem for a pure
Python package like SymPy but for e.g. NumPy or python-flint this
means producing almost twice as many binaries and testing twice as
many combinations. The most recent release of python-flint has
binaries for the Cartesian product of

CPython: 3.11, 3.12, 3.13, 3.13t (free-threaded)

OS/Arch:
MacOS x86_64
MacOS ARM
Windows x86_64
ManyLinux x86_64
ManyLinux ARM

That's 4*5 = 20 wheels with each at 10MB meaning that a full release
is 200MB of wheels on PyPI:
https://pypi.org/project/python-flint/#files

Supporting all non-EOL Python versions plus free-threaded would
increase the first part in future to:

CPython 3.13, 3.13t, 3.14, 3.14t, 3.15, 3.15t, 3.16, 3.16t, 3.17, 3.17t

Then it is 10*5 = 50 wheels making 0.5GB per release. PyPI has a limit
of 10GB for all releases of a project:
https://github.com/pypi/warehouse/blob/8060bfa3cb00c7f68bb4b10021b5361e92a04017/warehouse/forklift/legacy.py#L70-L72

At 0.5GB per release python-flint would exceed the file size limit
after 20 releases and would then need to start deleting older
releases.

Another thing on the horizon is Windows ARM and then there are also
other things like musllinux and so on that e.g. numpy provides wheels
for:
https://pypi.org/project/numpy/#files
Looks like NumPy has 50 wheels per release just for Python 3.10-3.13
and 3.13t already.

On top of 50 platforms to produce binaries then you can imagine
needing to test all of those with say a range of different versions of
something else like different SciPy versions or something.

Of course these file size limits and the combinatorial explosion of
platforms and ABIs are not directly relevant to SymPy but it shows why
other scientific Python packages don't want to say "let's just support
every non-EOL Python".

What matters for SymPy though is how its version support lines up with
other packages because as I said in the start the problem is that any
library that depends on SymPy without using an upper cap like
sympy<=3.13 is potentially broken if a new SymPy release goes out for
older Python versions that the dependent library no longer supports.

There is a general suggestion that Python packages should not use
upper cap version ranges like sympy<=3.13 but that only works if there
is some synchronisation around the supported Python versions.
Effectively it is the minimum Python versions required by each package
that acts as an implicit upper cap between e.g. SymPy and anything
else that depends on SymPy as well as all the things that SymPy
implicitly depends on like NumPy etc. If SymPy didn't specify a
minimum Python version then each new release would probably break
other packages for people using older Python versions.

> At the same time, I'm not aware of new Python features that we really
> wish we could be using but can't as long as we keep supporting older
> versions. Are there examples of this?

There aren't many new features that are needed in the sense that older
versions have to be dropped in order to get those features. Typically
this only happens for new syntax and recently I think all syntax
changes have been related to typing e.g. the syntax here would be
useful but cannot be used without dropping Python 3.11:
https://peps.python.org/pep-0695/

The question is not how hard or easy it is for SymPy to support many
Python versions or whether there are new features that would benefit
SymPy though. It is really about the fact that pushing a new SymPy
release out to an old version of Python risks breaking things that
depend on SymPy. If most other things follow SPEC 0 then it is better
that SymPy does as well but I'm not sure that is the case.

Anyway for now I think we are all agreed that SymPy 1.14 should stick
with 3.9 upwards.

--
Oscar

[Aaron Meurer]

What versions of Python does python-flint support? A potential issue
could be if python-flint follows SPEC 0 but SymPy supports older
Python versions, then SymPy would necessarily have to also support
older versions of python-flint, even if we might otherwise be able to
avoid that by doing releases in tandem.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/sympy/CAHVvXxTnD59ECAWDXf1GuO-73OUHfSZ30vtvkSECvoyUHTVRjg%40mail.gmail.com.

[Oscar Benjamin]

On Wed, 16 Apr 2025 at 17:28, Aaron Meurer <asme...@gmail.com> wrote:
>
> What versions of Python does python-flint support? A potential issue
> could be if python-flint follows SPEC 0 but SymPy supports older
> Python versions, then SymPy would necessarily have to also support
> older versions of python-flint, even if we might otherwise be able to
> avoid that by doing releases in tandem.

Currently python-flint requires Python 3.11, 3.12 or 3.13 but this is
mainly just due to the way providing binaries on PyPI works. Basically
if we don't put up binaries for 3.10 but the sdist says it works on
3.10 then anyone doing pip install python-flint will see pip trying to
build it from source which will almost certainly fail. The minimum
version is just set as whatever is the minimum that binaries will be
provided for.

SymPy 1.13 and 1.14 will allow any version of python-flint from 0.6 to
0.9 to be picked up implicitly (note that 0.7.1 is the most recent
version):
https://github.com/sympy/sympy/blob/089edf0ea752497120cbaa1a5af4b5d4bcab9a85/sympy/external/gmpy.py#L92-L96
If there is a different version then it just won't be used since it is
an optional dependency. It might be worth moving the upper version
limit to 0.10.0 for SymPy 1.14. The upper limit is just there so that
if python-flint does need to make a breaking change then there is some
way to do it without breaking older SymPy versions.

I suppose that if conda-forge adds python-flint as a hard dependency
for SymPy then it would either need to have different python-flint
versions for different sympy versions or otherwise conda-forge could
provide binaries for more versions. I don't think conda-forge cares
about the size/cost of having a huge platform matrix. Note that
currently python-flint 0.7 doesn't build in conda-forge:
https://github.com/conda-forge/python-flint-feedstock/pull/46

--
Oscar

[Aaron Meurer]

On Wed, Apr 16, 2025 at 11:14 AM Oscar Benjamin
<oscar.j....@gmail.com> wrote:
>
> On Wed, 16 Apr 2025 at 17:28, Aaron Meurer <asme...@gmail.com> wrote:
> >
> > What versions of Python does python-flint support? A potential issue
> > could be if python-flint follows SPEC 0 but SymPy supports older
> > Python versions, then SymPy would necessarily have to also support
> > older versions of python-flint, even if we might otherwise be able to
> > avoid that by doing releases in tandem.
>
> Currently python-flint requires Python 3.11, 3.12 or 3.13 but this is
> mainly just due to the way providing binaries on PyPI works. Basically
> if we don't put up binaries for 3.10 but the sdist says it works on
> 3.10 then anyone doing pip install python-flint will see pip trying to
> build it from source which will almost certainly fail. The minimum
> version is just set as whatever is the minimum that binaries will be
> provided for.

OK, my concern was more about if it would be hard for SymPy to support
multiple versions of python-flint and we had to pin versions, but it
doesn't sound like that is the case.

Aaron Meurer

>
> SymPy 1.13 and 1.14 will allow any version of python-flint from 0.6 to
> 0.9 to be picked up implicitly (note that 0.7.1 is the most recent
> version):
> https://github.com/sympy/sympy/blob/089edf0ea752497120cbaa1a5af4b5d4bcab9a85/sympy/external/gmpy.py#L92-L96
> If there is a different version then it just won't be used since it is
> an optional dependency. It might be worth moving the upper version
> limit to 0.10.0 for SymPy 1.14. The upper limit is just there so that
> if python-flint does need to make a breaking change then there is some
> way to do it without breaking older SymPy versions.
>
> I suppose that if conda-forge adds python-flint as a hard dependency
> for SymPy then it would either need to have different python-flint
> versions for different sympy versions or otherwise conda-forge could
> provide binaries for more versions. I don't think conda-forge cares
> about the size/cost of having a huge platform matrix. Note that
> currently python-flint 0.7 doesn't build in conda-forge:
> https://github.com/conda-forge/python-flint-feedstock/pull/46
>
> --
> Oscar
>

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/sympy/CAHVvXxSB2kH936qXSBKUd%3D3J4y-tNG6qQXu-hPCYOoaMUgDbZw%40mail.gmail.com.

[Amir Ebrahimi]

On Wednesday, April 16, 2025 at 5:22:41 AM UTC-7 Oscar wrote:

The question is not how hard or easy it is for SymPy to support many
Python versions or whether there are new features that would benefit
SymPy though. It is really about the fact that pushing a new SymPy
release out to an old version of Python risks breaking things that
depend on SymPy. If most other things follow SPEC 0 then it is better
that SymPy does as well but I'm not sure that is the case.

One suggestion would be to rev the major version number whenever you are going to release a version of SymPy that drops an older version of Python. I'd gather that most projects use SemVer with a version range that stays within a major version, so that'd prevent breakage in projects that are on an older version of Python. 

[Oscar Benjamin]

Pretty much every release of SymPy drops an older version of Python.
Nothing downstream (in terms of libraries) caps sympy<2 so the major
version bump would not prevent any breakage.

--
Oscar