Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages

[Matthias Koeppe]

We added the packages as optional "pip" packages (see https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#package-types for the terminology), each more than 1 year ago.

- https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest#spkg-pytest (added in 2020)

- https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_mock#spkg-pytest-mock (added in 2022)

- https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_xdist#spkg-pytest-xdist (added in 2022)

"pytest" is the current gold standard for running tests of Python packages. Various standard packages in the Sage distribution already declare pytest in "dependencies_check" as a conditional dependency for use when SAGE_CHECK=yes is set. The support for testing parts of the Sage library with pytest was from an effort largely stalled after 2022 -- and which has been completely broken since early 2024 with the arrival of pytest 8.x, which I have just fixed in https://github.com/sagemath/sage/pull/37999 (to be merged). I'll note that recent versions of pytest have added support for PEP-420 implicit namespace package, which the Sage library uses as part of the modularization effort.

By making pytest a standard package, I would hope to help revive this effort, and thus the larger effort to "adopt mainstream Python testing/linting infrastructure" (see https://github.com/sagemath/sage/issues/28936). 

I'm proposing to make the packages (and their dependencies) standard (wheel) packages according to the procedures in our developer guide. 

- Reference to previous such proposals following our project's procedures: https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/KyOOQzkzAAAJ

- Link to our packaging documentation and explanation how making it a standard package compares to version pinning done for example using conda-lock:  https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/N2vuKb-TAAAJ

The other pytest_* packages are related technical packages.

The PR that implements it: https://github.com/sagemath/sage/pull/37301

This is a redo of the 2nd part of my proposal https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/NsyUa7iXAgAJ from Feb 10, which was stalled. (The 1st part (on python_build) was redone in https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/9_zh6usoAAAJ and approved.)

I ask everyone to focus on the specifics of this proposal.

[Dima Pasechnik]

My objections (and not only my) to this still stand - and, frankly, I
don't see anything new here. What's the point of bringing the stalled
matters up again in this original unchanged way ?

We don't need to have all the dependencies of packages like pytest in
Sage, it's just pure bloat, give their peripheral role in particular.
That is, it's fine to declare them standard, and keep them pip
packages - what do we lose this way? Nothing, and we don't bloat Sage
with even more packages nobody knows anything about - besides them
being dependencies of something in Sage.

But there is more trouble ahead if the currect proposal gets in over
my objections. Say, the next version of pytest might get a part which
needs Rust, and pytest is a wheel package, with all its buildable from
source dependencies in Sage, and Sage is fully committed to using
pytest for testing.
Are you going to include a Rust toolchain in Sage ? No, obviously not.
Are you going to demote pytest back to optional,
and throw away work done on using pytest more? No. Have another fight
over the ways Sage should be packaged? Yes, sure.

I think the only feasible way forward here is as I proposed (standard
pip packages), and I propose it again.

Dima

>
>
> I ask everyone to focus on the specifics of this proposal.
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/2c42bd41-24a3-467e-857f-aedc3966c107n%40googlegroups.com.

[David Roe]

I don't find it plausible that a package whose purpose is to test Python code would add Rust as a dependency.  Sure, dependencies can change as projects evolve and add features, but I don't think it's reasonable to base decisions on what to add to the sage distribution on potential future changes.

David

I think the only feasible way forward here is as I proposed (standard
pip packages), and I propose it again.

Dima

>
>
> I ask everyone to focus on the specifics of this proposal.
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/2c42bd41-24a3-467e-857f-aedc3966c107n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq2fubG32GKjKroEVJ_LKB2WUY%2BqJSXCZTgd0ROC%3DYHczA%40mail.gmail.com.

[Matthias Koeppe]

On Friday, May 31, 2024 at 9:38:30 AM UTC-7 Dima Pasechnik wrote:

frankly, I don't see anything new here. 

There does not have to be anything new.

The proposal stands on its own merit.

[dim...@gmail.com]

We already are using a Rust-based tool to improve Sage code, namely,
ruff (https://pypi.org/project/ruff/)
https://github.com/sagemath/sage/pulls?q=is%3Apr+is%3Aopen+ruff

> Sure, dependencies can change as
> projects evolve and add features, but I don't think it's reasonable to base
> decisions on what to add to the sage distribution on potential future
> changes.

Well, Rust or no Rust: if you act myopically, you're doomed to do extra
needless work/pay extra money. There are other reasons not to add extra wheel
packages if it can be avoided: distribution bandwidth might stop being free,
disk space might stop being free, other "too big to be sustainable"
issues are popping up - we're having so many packages that they already
scream for a proper package manager, and not a slow kludge we have
instead.

Dima

> David
>
> I think the only feasible way forward here is as I proposed (standard
> > pip packages), and I propose it again.
> >
> > Dima
> >
> > >
> > >
> > > I ask everyone to focus on the specifics of this proposal.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "sage-devel" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > an email to sage-devel+...@googlegroups.com.
> > > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/sage-devel/2c42bd41-24a3-467e-857f-aedc3966c107n%40googlegroups.com
> > .
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "sage-devel" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to sage-devel+...@googlegroups.com.
> > To view this discussion on the web visit

> > https://groups.google.com/d/msgid/sage-devel/CAAWYfq2fubG32GKjKroEVJ_LKB2WUY%2BqJSXCZTgd0ROC%3DYHczA%40mail.gmail.com
> > .

> >
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAChs6_kCgBRXMZWtptJPfDpOqbxTXF2gdeQJ8%3D-oNGRrEv%2BVdw%40mail.gmail.com.

[Nathan Dunfield]

The six packages proposed for addition are all pure-Python and collectively take up 4M installed (about a 450K download); for scale, I think a full Sage installation is about 3G (1.25G download).  They all seem to be well-maintained, and common enough that all are available on conda-forge.  Using pytest to test Sage sounds like a good thing to me, and I think it makes sense to promote them standard packages.

Best,

Nathan

[Dima Pasechnik]

I don't argue against making them standard - I argue against vendoring them, which buys us essentially nothing but an extra headache.

[Matthias Koeppe]

I'll share some additional facts for everyone's convenience.

The total size of these 5 wheel packages to be added in https://github.com/sagemath/sage/pull/37301is about 500 kilobytes. (As a comparison, that's 10% of the size of our "configure" script.)

-rw-r--r--  1 mkoeppe  staff   40612 May 22 12:47 upstream/execnet-2.1.1-py3-none-any.whl
-rw-r--r--  1 mkoeppe  staff    5892 May 22 12:57 upstream/iniconfig-2.0.0-py3-none-any.whl
-rw-r--r--  1 mkoeppe  staff  339593 May 22 12:48 upstream/pytest-8.2.1-py3-none-any.whl
-rw-r--r--  1 mkoeppe  staff    9863 May 22 12:47 upstream/pytest_mock-3.14.0-py3-none-any.whl
-rw-r--r--  1 mkoeppe  staff   46108 May 22 12:48 upstream/pytest_xdist-3.6.1-py3-none-any.whl

On Thursday, May 30, 2024 at 3:25:08 PM UTC-7 Matthias Koeppe wrote:

I ask everyone to focus on the specifics of this proposal.

In particular, I'll suggest to refrain from engaging with generic warnings about "bloat", hypothetical scenarios about expensive bandwidth and storage, "backdooring", etc.

[dim...@gmail.com]

Haven't it been discussed already?

I repeat: there is no point in having pytest and its dependencies vendored
in Sage. pytest can be kept a pip package, just promoted to standard.
Unless we're trying to be a (bad) mirror of PyPI.

Dima

>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/6014f157-a90e-4e6d-af94-05aa62b8e6fcn%40googlegroups.com.

[Matthias Koeppe]

On Monday, June 3, 2024 at 12:29:14 PM UTC-7 dim...@gmail.com wrote:

pytest can be kept a pip package, just promoted to standard.

It cannot, per existing policy.

And the policy exists because it was made in awareness of the limitations of "pip" packages.

[Matthias Koeppe]

All, please send your comments on this proposal.

I hope that we can count votes here by the end of the week.

On Thursday, May 30, 2024 at 3:25:08 PM UTC-7 Matthias Koeppe wrote:

[David Roe]

I see the policy you refer to here, but what limitations of pip packages prevent them from being standard?

David

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/4be6a58d-8fb6-45f0-84ae-c689b43e4502n%40googlegroups.com.

[Matthias Koeppe]

On Wednesday, June 5, 2024 at 1:38:39 PM UTC-7 David Roe wrote:

On Wed, Jun 5, 2024 at 4:25 PM Matthias Koeppe <matthia...@gmail.com> wrote:

And the policy exists because it was made in awareness of the limitations of "pip" packages.

I see the policy you refer to here, but what limitations of pip packages prevent them from being standard?

This discussion does not belong into this thread, but I'll explain a bit of this in the other thread.

[Matthias Koeppe]

I consider this approved. 

The PR is open for review at https://github.com/sagemath/sage/pull/37301

On Thursday, May 30, 2024 at 3:25:08 PM UTC-7 Matthias Koeppe wrote: