99 views

Skip to first unread message

Jan 13, 2021, 10:41:48 PMJan 13

to sage-devel

Meta-ticket https://trac.sagemath.org/ticket/31164 proposes to add user packages, such as those listed at https://wiki.sagemath.org/SageMathExternalPackages, to the Sage distribution as "pip" packages. This has the following benefits:

- They will be automatically included in our reference manual and Sage website (https://trac.sagemath.org/ticket/29655).
- We have GitHub Actions workflows in place (tox-optional.yml, tox-experimental.yml), which will automatically try to build the packages on each beta release. This provides continuous integration that will allow us to catch unintended breaking changes during the Sage development cycle.

- If a package listed at https://wiki.sagemath.org/SageMathExternalPackages is outdated (does not work with a current version of Sage), it will be helpful to add a note to the wiki page and to notify the package authors.

- If a package seems to work with current Sage, help by creating a ticket for the inclusion in Sage and list the ticket in meta-ticket https://trac.sagemath.org/ticket/31164

- If a significant package is neither listed in build/pkgs/ nor in the wiki page, please add it.

Feb 7, 2021, 7:49:23 PMFeb 7

to sage-devel

The first few user packages have been added in 9.3.beta7:

ore_algebra, sage_flatsurf, admcycles, slabbe, surface_dynamics.

Feb 8, 2021, 2:55:22 AMFeb 8

to sage-...@googlegroups.com

Great!

Should they be removed from the wiki?

(https://wiki.sagemath.org/SageMathExternalPackages)

The list of "official" user packages are now part of the sage

documentation. We could keep the wiki for the not yet integrated

user packages.

Le 08/02/2021 à 01:49, Matthias Koeppe a écrit :

> The first few user packages have been added in 9.3.beta7:

> ore_algebra, sage_flatsurf, admcycles, slabbe, surface_dynamics.

>

>

> On Wednesday, January 13, 2021 at 7:41:48 PM UTC-8 Matthias Koeppe wrote:

>

>> Meta-ticket https://trac.sagemath.org/ticket/31164 proposes to add user

>> packages, such as those listed at

>> https://wiki.sagemath.org/SageMathExternalPackages, to the Sage

>> distribution as "pip" packages. This has the following benefits:

>>

>> - They will be automatically included in our reference manual and Sage

>> website (https://trac.sagemath.org/ticket/29655).

>> - We have GitHub Actions workflows in place

Should they be removed from the wiki?

(https://wiki.sagemath.org/SageMathExternalPackages)

The list of "official" user packages are now part of the sage

documentation. We could keep the wiki for the not yet integrated

user packages.

Le 08/02/2021 à 01:49, Matthias Koeppe a écrit :

> The first few user packages have been added in 9.3.beta7:

> ore_algebra, sage_flatsurf, admcycles, slabbe, surface_dynamics.

>

>

> On Wednesday, January 13, 2021 at 7:41:48 PM UTC-8 Matthias Koeppe wrote:

>

>> Meta-ticket https://trac.sagemath.org/ticket/31164 proposes to add user

>> packages, such as those listed at

>> https://wiki.sagemath.org/SageMathExternalPackages, to the Sage

>> distribution as "pip" packages. This has the following benefits:

>>

>> website (https://trac.sagemath.org/ticket/29655).

>> - We have GitHub Actions workflows in place

Feb 8, 2021, 7:06:38 AMFeb 8

to sage-...@googlegroups.com

Hi,

On Sun, Feb 07, 2021 at 04:49:23PM -0800, Matthias Koeppe wrote:

> The first few user packages have been added in 9.3.beta7:

> ore_algebra, sage_flatsurf, admcycles, slabbe, surface_dynamics.

We should notice that, contrary to other packages, those packages should

be considered as downstream (not upstream), and this should be reflected

in our release process.

Indeed, their code get adapted after Sage changes, see e.g. (randomly

chosen examples):

https://github.com/mkauers/ore_algebra/commit/2d71b50ebad81e62432482facfe3f78cc4961c4f

https://github.com/mkauers/ore_algebra/commit/8c678f9c70aee8df211c04211dd2f7e230a4770f

For upstream packages (especially important ones), they should usually

be merged in the beginning of a new release (on beta[n] with small n),

so that incompatibilities could be reported and fixed before the

official release.

In the current case, it should be the opposite, after downstream adapted

their code to our changes, so that those packages are not broken for the

official release.

Hence, there should be a way to update the versions of such packages

during beta[-1] (the one before rc0), at least those packages should be

tagged "downstream" somewhere (e.g. whithin SPKG.rst file or whatever),

so that someone could have a look before the rc.

While our naming of development releases is pretty poor (beta or rc), we

should perhaps indicate somewhere which intervals are reserved for

backward-incompatible changes, and so on.

Another question: what happens if at the end of a Sage release cycle,

the downstream package is still broken ? Shall we remove the package

from build/pkgs ? Shall we add a temporary patch within the spkg

directory ? Or add a "broken" keyword for the "type" file ?

Ciao,

Thierry

> On Wednesday, January 13, 2021 at 7:41:48 PM UTC-8 Matthias Koeppe wrote:

>

> > Meta-ticket https://trac.sagemath.org/ticket/31164 proposes to add user

> > packages, such as those listed at

> > https://wiki.sagemath.org/SageMathExternalPackages, to the Sage

> > distribution as "pip" packages. This has the following benefits:

> >

> > - They will be automatically included in our reference manual and Sage

> > website (https://trac.sagemath.org/ticket/29655).

> > - We have GitHub Actions workflows in place

> You received this message because you are subscribed to the Google Groups "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/88ba0229-2e6f-4a04-a955-157248c5f90bn%40googlegroups.com.

On Sun, Feb 07, 2021 at 04:49:23PM -0800, Matthias Koeppe wrote:

> The first few user packages have been added in 9.3.beta7:

> ore_algebra, sage_flatsurf, admcycles, slabbe, surface_dynamics.

be considered as downstream (not upstream), and this should be reflected

in our release process.

Indeed, their code get adapted after Sage changes, see e.g. (randomly

chosen examples):

https://github.com/mkauers/ore_algebra/commit/2d71b50ebad81e62432482facfe3f78cc4961c4f

https://github.com/mkauers/ore_algebra/commit/8c678f9c70aee8df211c04211dd2f7e230a4770f

For upstream packages (especially important ones), they should usually

be merged in the beginning of a new release (on beta[n] with small n),

so that incompatibilities could be reported and fixed before the

official release.

In the current case, it should be the opposite, after downstream adapted

their code to our changes, so that those packages are not broken for the

official release.

Hence, there should be a way to update the versions of such packages

during beta[-1] (the one before rc0), at least those packages should be

tagged "downstream" somewhere (e.g. whithin SPKG.rst file or whatever),

so that someone could have a look before the rc.

While our naming of development releases is pretty poor (beta or rc), we

should perhaps indicate somewhere which intervals are reserved for

backward-incompatible changes, and so on.

Another question: what happens if at the end of a Sage release cycle,

the downstream package is still broken ? Shall we remove the package

from build/pkgs ? Shall we add a temporary patch within the spkg

directory ? Or add a "broken" keyword for the "type" file ?

Ciao,

Thierry

> On Wednesday, January 13, 2021 at 7:41:48 PM UTC-8 Matthias Koeppe wrote:

>

> > Meta-ticket https://trac.sagemath.org/ticket/31164 proposes to add user

> > packages, such as those listed at

> > https://wiki.sagemath.org/SageMathExternalPackages, to the Sage

> > distribution as "pip" packages. This has the following benefits:

> >

> > website (https://trac.sagemath.org/ticket/29655).

> > - We have GitHub Actions workflows in place

> > (tox-optional.yml, tox-experimental.yml), which will automatically try to

> > build the packages on each beta release. This provides continuous

> > integration that will allow us to catch unintended breaking changes during

> > the Sage development cycle.

> >

> > Help is needed with this task.

> > - If a package listed at

> > https://wiki.sagemath.org/SageMathExternalPackages is outdated (does not

> > work with a current version of Sage), it will be helpful to add a note to

> > the wiki page and to notify the package authors.

> > - If a package seems to work with current Sage, help by creating a ticket

> > for the inclusion in Sage and list the ticket in meta-ticket

> > https://trac.sagemath.org/ticket/31164

> > - If a significant package is neither listed in build/pkgs/ nor in the

> > wiki page, please add it.

> >

> >

> >

> >

> >

> >

>

> --
> > build the packages on each beta release. This provides continuous

> > integration that will allow us to catch unintended breaking changes during

> > the Sage development cycle.

> >

> > Help is needed with this task.

> > - If a package listed at

> > https://wiki.sagemath.org/SageMathExternalPackages is outdated (does not

> > work with a current version of Sage), it will be helpful to add a note to

> > the wiki page and to notify the package authors.

> > - If a package seems to work with current Sage, help by creating a ticket

> > for the inclusion in Sage and list the ticket in meta-ticket

> > https://trac.sagemath.org/ticket/31164

> > - If a significant package is neither listed in build/pkgs/ nor in the

> > wiki page, please add it.

> >

> >

> >

> >

> >

> >

>

> You received this message because you are subscribed to the Google Groups "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/88ba0229-2e6f-4a04-a955-157248c5f90bn%40googlegroups.com.

Feb 8, 2021, 7:19:17 AMFeb 8

to sage-...@googlegroups.com

For most of the packages, there is no explicit version tracking. On

the current rather small list only ore_algebra has an explicit

reference to a commit in requirements.txt

$ cat admcycles/requirements.txt

admcycles

$ cat surface_dynamics/requirements.txt

surface_dynamics

$ cat ore_algebra/requirements.txt

git+https://github.com/mkauers/ore_algebra@6826ac49b4cdf66a563449aced21a2fd1fd085c9#egg=ore_algebra

If instead ore_algebra would properly release on PyPI, the

sage side of things would be simpler.

Best

Vincent

PS: still there is the issue of "old" SageMath versions that might

not be compatible with "recent" package versions. The only mechanism

is to tight a single version of a package to a SageMath version as it

was done with ore_algebra. The latter is quite restrictive given that

packges might evolve between two sage releases.

the current rather small list only ore_algebra has an explicit

reference to a commit in requirements.txt

$ cat admcycles/requirements.txt

admcycles

$ cat surface_dynamics/requirements.txt

surface_dynamics

$ cat ore_algebra/requirements.txt

git+https://github.com/mkauers/ore_algebra@6826ac49b4cdf66a563449aced21a2fd1fd085c9#egg=ore_algebra

If instead ore_algebra would properly release on PyPI, the

sage side of things would be simpler.

Best

Vincent

PS: still there is the issue of "old" SageMath versions that might

not be compatible with "recent" package versions. The only mechanism

is to tight a single version of a package to a SageMath version as it

was done with ore_algebra. The latter is quite restrictive given that

packges might evolve between two sage releases.

Feb 8, 2021, 1:34:21 PMFeb 8

to sage-devel

On Sunday, February 7, 2021 at 11:55:22 PM UTC-8 vdelecroix wrote:

Should they be removed from the wiki?

(https://wiki.sagemath.org/SageMathExternalPackages)

The list of "official" user packages are now part of the sage

documentation. We could keep the wiki for the not yet integrated

user packages.

I would suggest that we remove them from that page when 9.3 is released.

Feb 8, 2021, 1:42:59 PMFeb 8

to sage-devel

On Monday, February 8, 2021 at 4:06:38 AM UTC-8 Thierry (sage-googlesucks@xxx) wrote:

On Sun, Feb 07, 2021 at 04:49:23PM -0800, Matthias Koeppe wrote:

> The first few user packages have been added in 9.3.beta7:

> ore_algebra, sage_flatsurf, admcycles, slabbe, surface_dynamics.

We should notice that, contrary to other packages, those packages should

be considered as downstream (not upstream), and this should be reflected

in our release process.

That's right, this is an important distinction.

In the current case, it should be the opposite, after downstream adapted

their code to our changes, so that those packages are not broken for the

official release.

Hence, there should be a way to update the versions of such packages

during beta[-1] (the one before rc0), at least those packages should be

tagged "downstream" somewhere (e.g. whithin SPKG.rst file or whatever),

so that someone could have a look before the rc.

As Vincent has pointed out, we currently do not use version constraints for these (downstream) user packages.

I think it should be the responsibility of the downstream package to define a versioning policy - define when they plan to drop support for a particular Sage version.

Then we can use an appropriate version constraint in the "install-requires.txt" file for this package in build/pkgs.

Another question: what happens if at the end of a Sage release cycle,

the downstream package is still broken ? Shall we remove the package

from build/pkgs ? Shall we add a temporary patch within the spkg

directory ? Or add a "broken" keyword for the "type" file ?

Setting type to "experimental" would be good enough, I think. This already issues a scary warning.

Feb 8, 2021, 1:44:03 PMFeb 8

to sage-devel

On Monday, February 8, 2021 at 10:42:59 AM UTC-8 Matthias Koeppe wrote:

I think it should be the responsibility of the downstream package to define a versioning policy - define when they plan to drop support for a particular Sage version.Then we can use an appropriate version constraint in the "install-requires.txt" file for this package in build/pkgs.

I meant "requirements.txt" actually.

Feb 8, 2021, 5:48:13 PMFeb 8

to sage-...@googlegroups.com

Hi,

On Mon, Feb 08, 2021 at 10:42:59AM -0800, Matthias Koeppe wrote:

[...]

packages, we are going to run into troubles, as we can not expect users

to use latest Sagemath version when they install some downstream

package.

To take the previous example:

https://github.com/mkauers/ore_algebra/commit/8c678f9c70aee8df211c04211dd2f7e230a4770f

without versionning on our side, if a user tries to install ore_algebra

after 7 November 2017 while using 8.0, they would get import errors.

Note also that this downstream change was done during the preparation of

8.1 (that was released on 7 December 2017), so even up-to-date

non-developer users would have been running 8.0 at that time and

therefore get errors.

On Mon, Feb 08, 2021 at 01:17:34PM +0100, Vincent Delecroix wrote:

[...]

providing downstream versions.

Ciao,

Thierry

On Mon, Feb 08, 2021 at 10:42:59AM -0800, Matthias Koeppe wrote:

[...]

> As Vincent has pointed out, we currently do not use version

> constraints for

> these (downstream) user packages.

I think that if we do not provide explicit version of downstream
> constraints for

> these (downstream) user packages.

packages, we are going to run into troubles, as we can not expect users

to use latest Sagemath version when they install some downstream

package.

To take the previous example:

https://github.com/mkauers/ore_algebra/commit/8c678f9c70aee8df211c04211dd2f7e230a4770f

without versionning on our side, if a user tries to install ore_algebra

after 7 November 2017 while using 8.0, they would get import errors.

Note also that this downstream change was done during the preparation of

8.1 (that was released on 7 December 2017), so even up-to-date

non-developer users would have been running 8.0 at that time and

therefore get errors.

On Mon, Feb 08, 2021 at 01:17:34PM +0100, Vincent Delecroix wrote:

[...]

> $ cat ore_algebra/requirements.txt

> git+https://github.com/mkauers/ore_algebra@6826ac49b4cdf66a563449aced21a2fd1fd085c9#egg=ore_algebra

>

> If instead ore_algebra would properly release on PyPI, the

> sage side of things would be simpler.

I do not see how having ore_algebra in PyPI would fix the issue of not
> git+https://github.com/mkauers/ore_algebra@6826ac49b4cdf66a563449aced21a2fd1fd085c9#egg=ore_algebra

>

> If instead ore_algebra would properly release on PyPI, the

> sage side of things would be simpler.

providing downstream versions.

Ciao,

Thierry

Feb 8, 2021, 8:22:18 PMFeb 8

to sage-devel

On Monday, February 8, 2021 at 10:42:59 AM UTC-8 Matthias Koeppe wrote:

I think it should be the responsibility of the downstream package to define a versioning policy - define when they plan to drop support for a particular Sage version.Then we can use an appropriate version constraint in the "install-requires.txt" file for this package in build/pkgs.

Another thing that a downstream package can do is declare an "install_requires" on the Sage library, specifying a supported range of versions. Then we should be able to use the dependency resolution by pip to be sure to install a compatible version.

Note https://trac.sagemath.org/ticket/30912 updates the metadata of the Sage library, defining the distribution name "sagemath-standard". (Another distribution that could be used for versioning is "sage_conf".)

Also note the pip dependency resolver has just changed with the update to pip 20.3.3 in #30589 (see https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020).

Feb 9, 2021, 9:11:01 AMFeb 9

to sage-...@googlegroups.com

What is the most recent version of package X that

runs on Sage version Y?

On the downstream side, one could just add a requirement

sagemath>=Y

meaning that sagemath < Y is not supported.

However, I don't know enough of pip to understand how to make

"pip install X" retrieve the information. Would that work out

of the box with sage 9.3.beta7?

Best

Vincent

Feb 27, 2021, 7:13:27 AMFeb 27

to sage-...@googlegroups.com

Hi Thierry,

On 2021-02-08, Thierry <sage-goo...@lma.metelu.net> wrote:

> We should notice that, contrary to other packages, those packages should

> be considered as downstream (not upstream), and this should be reflected

> in our release process.

>

> Indeed, their code get adapted after Sage changes, see e.g. (randomly

> chosen examples):

>

> https://github.com/mkauers/ore_algebra/commit/2d71b50ebad81e62432482facfe3f78cc4961c4f

> https://github.com/mkauers/ore_algebra/commit/8c678f9c70aee8df211c04211dd2f7e230a4770f

Then, I think, p_group_cohomology should be considered downstream, too.

Most of its latest releases were triggered by changes in Sage.

Best regards,

Simon

On 2021-02-08, Thierry <sage-goo...@lma.metelu.net> wrote:

> We should notice that, contrary to other packages, those packages should

> be considered as downstream (not upstream), and this should be reflected

> in our release process.

>

> Indeed, their code get adapted after Sage changes, see e.g. (randomly

> chosen examples):

>

> https://github.com/mkauers/ore_algebra/commit/2d71b50ebad81e62432482facfe3f78cc4961c4f

> https://github.com/mkauers/ore_algebra/commit/8c678f9c70aee8df211c04211dd2f7e230a4770f

Most of its latest releases were triggered by changes in Sage.

Best regards,

Simon

Mar 7, 2021, 4:15:11 PMMar 7

to sage-devel

9.3.beta8 adds:

snappy

Reply all

Reply to author

Forward

0 new messages

Search

Clear search

Close search

Google apps

Main menu