drop python2 compatibility in 9.1 ?

[Frédéric Chapoton]

Hello,

I would like to suggest that the sooner we drop Python 2 support the better. We still need to handle the upgrade to ipython7 and the compatibility with python 3.8. All this will be made very difficult if we insist on maintaining a codebase that is both compatible with python 2 and python 3.

So, please vote :

Do you agree that sage release 9.1 (and most of the 9.1.betas) will not be kept compatible with Python 2 ?

Frédéric

[François Bissey]

[x] drop python 2.7 compatibility

[Eric Gourgoulhon]

+1 for dropping Python 2 compatibility in Sage 9.1

Eric.

[Nils Bruin]

I think our wiki vetoes that idea. See https://wiki.sagemath.org/Python3-Switch :

Compiling with Python 2

After version 9.0, if you really want so, you can still build and use SageMath with Python 2, as follows.

make configure
./configure --with-python=2
make build

Beware that you will need to call the second line again if you ever call "make distclean".

This will work until version 9.1 at least. Then the backward compatibility with Python 2 will no longer be ensured.

If you want to drop py2 compatibility very soon, the only option is to release 9.1 basically right now, identical to 9.0, and then get on with developing 9.2. That's a nasty thing to do. Based on previous release schedules, people would be justified in expecting that <=9.1 is the "current" release until at least June 2020 or so. So we're stuck with py2 compatibility until that time.

[John H Palmieri]

Can someone with trac admin access and know-how add a 9.2 milestone, so people can develop for 9.2 if they want to focus on dropping Python 2 support?

Meanwhile, 9.1 should include a deprecation warning when people use './configure --with-python=2'.

[David Roe]

I just added a 9.2 milestone on trac, which isn't the default.

David

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/2a6cdc41-1ef1-4f95-b44c-c062342dc82b%40googlegroups.com.

[Timo Kaufmann]

+1 for dropping python 2.7 compatibility

It is unfortunate that the wiki makes that promise. But it is a community wiki, and as such I wouldn't consider it an authorative source. Keeping python2 support will be a *lot* of effort, either for the sage project or (if it just refuses to deal with this problem and keeps using outdated dependencies) for packagers. At the same time, I see little to no benefit in keeping it. People who insist on python2 can keep using 9.0 as long as they need to. It will continue to work for all the purposes it works for now.

[Nils Bruin]

On Sunday, January 5, 2020 at 2:33:02 PM UTC-8, Timo Kaufmann wrote:

+1 for dropping python 2.7 compatibility

It is unfortunate that the wiki makes that promise. But it is a community wiki, and as such I wouldn't consider it an authorative source. Keeping python2 support will be a *lot* of effort, either for the sage project or (if it just refuses to deal with this problem and keeps using outdated dependencies) for packagers. At the same time, I see little to no benefit in keeping it. People who insist on python2 can keep using 9.0 as long as they need to. It will continue to work for all the purposes it works for now.

I don't think there's a doubt that we'll drop py2.7 support. This Wiki, however, has been referred to in many places describing the py2/py3 transition. As far as I'm aware this is the only announcement at the moment that comes close to an official py2/py3 transition plan for sage. If you're aware of a message elsewhere, we can see how we can unify the two. The message on the wiki makes a very definite mention that 9.1 will still support building with python2. I don't consider reneging on that an option. Breaking definite promises in support makes for *very* bad PR. If there is a more "official" place to mention the transition strategy, we can put an announcement there, but given the fact that we've already published the message on the wiki and pointed at it means that that announcement must be as least as lenient wrt. py2 as the wiki message.

It's not such a burden for developers anyway: you can just develop for Py3 only. Your work will just be merged in 9.2 at the earliest. If we end up with loads of 9.2 milestone tickets and no 9.1 ones, releasing 9.1 will be quite straightforward.

[Nils Bruin]

On Sunday, January 5, 2020 at 2:51:44 PM UTC-8, Nils Bruin wrote:

I don't think there's a doubt that we'll drop py2.7 support. This Wiki, however, has been referred to in many places describing the py2/py3 transition.

One of these places is the FAQ, which is official documentation:

http://doc.sagemath.org/html/en/faq/faq-usage.html#can-i-use-sagemath-with-python-3-x

The 9.0 FAQ still mentions:

As of August 2019, most of SageMath works fine with Python 3. However, we still consider Python 3 support to be experimental and no official Python 3 release has been made yet.

That would probably be good to adjust too: there is now an official Py3 release and I don't think we consider it experimental anymore -- unless sage in its entirety is still considered experimental :-).

[Andrey Novoseltsev]

I fail to see the difference between "officialness" of this documentation statement that Python 3 support is experimental and the wiki promise that 9.1 will work with Python 2. If the first one is wrong and it is OK to correct it and move forward, why can't the second one also be corrected?

Those who do want to keep that promise intact - can you please explain who will be affected and how if we drop Python 2 support? "Regular" users do not have a choice - precompiled binaries are with Python 3 only. Those who can build from source and add configure parameters may just as well checkout 9.0 and compile that.

Note that I am not a particularly active Sage developer anymore - I maintain SageMathCell, which is switched to Python 3 already. So it does not affect me and my workload in any way whether 9.1 is Python 2 compatible or not. But for the sake of those who do work on transition - please don't force them to spend their valuable energy on a pretty much pointless compatibility with unsupported languages.

[Matthias Koeppe]

+1 for dropping python 2 support in 9.1 betas or 9.2 betas if the 9.1 cycle is not too long.

Matthias

[Jori Mäntysalo (TAU)]

On Sun, 5 Jan 2020, Frédéric Chapoton wrote:

> Do you agree that sage release 9.1 (and most of the 9.1.betas) will not be
> kept compatible with Python 2 ?

I agree.

--
Jori Mäntysalo

Tampereen yliopisto - Ihminen ratkaisee

[Nils Bruin]

On Sunday, January 5, 2020 at 7:53:35 PM UTC-8, Andrey Novoseltsev wrote:

Those who do want to keep that promise intact - can you please explain who will be affected and how if we drop Python 2 support? "Regular" users do not have a choice - precompiled binaries are with Python 3 only. Those who can build from source and add configure parameters may just as well checkout 9.0 and compile that.

You can see the problem with the py2/py3 transition already in this asksage question about sagecell: https://ask.sagemath.org/question/49335/deprecationwarning-on-sagecell-python-3/ :

The transition to py3 instantly invalidated a whole bunch of sage-cell contents, because there are sufficiently many differences between py2 and py3 that almost any code needs some (fairly routine) changes. Just a with deprecation in the user interface, we must have a transition period that gives people a chance to change over: where there's a relatively straightforward way to still run the py2 code unchanged, so that the user can plan for an opportune time to make the required changes.

You could say: there is such a way: just check out 9.0 and build that with py2. Perhaps that's enough. However, I would err at the side of caution and make a slightly bigger gesture here and still include 9.1 as well -- that was published as a definite promise already. Once again, for any piece of code where this turns out to be a bother, just wait with the merge until 9.2. I don't think it needs to put a very large load on developers. It will mainly be a pain for the release manager, so he/she should actually get to have a significant voice in this.

We're late to the game already with making the transition only at the point where py2 has fallen out of support.

In fact, I think it would make a lot of sense to make 9.1 the last release to support py2 and therefore make 9.1 more a bugfix release: concentrate on fixes of issues that arise as a result of 9.0 and fix them, so that there's a relatively reliable point in the history of sage to run on py2. That would automatically target significant new features for 9.2.

[Antonio Rojas]

El lunes, 6 de enero de 2020, 8:43:14 (UTC+1), Nils Bruin escribió:

You could say: there is such a way: just check out 9.0 and build that with py2. Perhaps that's enough. However, I would err at the side of caution and make a slightly bigger gesture here and still include 9.1 as well -- that was published as a definite promise already. Once again, for any piece of code where this turns out to be a bother, just wait with the merge until 9.2. I don't think it needs to put a very large load on developers. It will mainly be a pain for the release manager, so he/she should actually get to have a significant voice in this

It's not as simple as waiting for 9.2 to merge them. Some of the pending changes (ipython 7 and python 3.8 support) require huge patches that touch lots of files. This means that they will probably need rebasing for every single beta release (this has been the case until now), which makes it not really practical to start working on them on trac until the 9.2 cycle starts. Until then the burden of maintaining the patches falls on the downstream packagers - and the situation is only going to get worse as new python3-only versions of dependencies are released and updated on distros. 

 

[Frédéric Chapoton]

the wiki promise was written by myself alone, and I think it should be changed. I think there will be very few (or  no) people that will desire to compile sage 9.0 with python 2. And 9.1 even more. People that want to be up to date do not care about py2.

I therefore strongly vote for the immediate drop of python2, and for removing the promise that 9.1 will be py2-compatible

F

[Simon King]

Hi Nils,

On 2020-01-06, Nils Bruin <nbr...@sfu.ca> wrote:
> In fact, I think it would make a lot of sense to make 9.1 the last release
> to support py2 and therefore make 9.1 more a bugfix release: concentrate on
> fixes of issues that arise as a result of 9.0 and fix them, so that there's
> a relatively reliable point in the history of sage to run on py2. That
> would automatically target significant new features for 9.2.

Having a bugfix release as the last py-2-supporting release sounds like
a good plan to me. So, +1 to this.

Also I think that the additional burden to average developers will be
small: If (on top of bug fixes) there would be new code in v9.1, then
making it work with py-2 would basically involve some "from __future__
import", isn't it?

Best regards,
Simon

[E. Madison Bray]

I agree with Nils. There should be at least a one release deprecation
period. Also, while I don't think we use any kind of real semantic
versioning, I think we should name a Python 3-only release 10.0 as
it's a very major backwards-incompatibility change.

Instead for 9.1 let us display a prominent deprecation message in any
Python 2 builds when running Sage, including a link to a guide for
porting existing code (for Sage there is not all that much to
do--mostly any generic Python 2 to 3 porting guide will do, plus some
Sage-specific caveats of which I can't think of many).

[kcrisman]

Because a lot of Sage users are also developers (presumably a higher proportion than for Python or Mathematica!), a lot of Sage work has had developers in mind.  However, I think it is pretty important to pay attention to the users, the vast majority of whom we don't ever hear from, since they aren't paying for Sage.

As an example, if there were a coordinated strategy that a company would make, I can't imagine that there wouldn't have been two versions of Sage cell server going at once - or even an addition (temporary) to the drop-down menu saying "Sage with Python 2", for transitioning. I don't know if I want to ask the very capable, but very busy, Andrey N. to do that!  But once again, sage-devel will never know, because the users didn't pay anything, so they may just decide to drop it.  I know it's a headache for any authors using PreTeXt who used "print" statements, because it's not just changing your own code, but releasing a new version of a book.  Presumably many many worksheets (sagenb or Jupyter) also are in the same boat - and one can say "oh, there was time" but we all know that many Sage users do not actually upgrade very frequently, because Sage is so usable and stable in the core functionality, so they may not have been at all aware of the necessity.  They just know their code is breaking.

This doesn't mean we shouldn't have switched to Py3 or Jupyter or whatever the backwards-incompatible flavor of the month is - that much is clear.  But the only way the community grows is by attracting people long enough to have a certain percentage become developers, and that does mean some boring "customer service" for our volunteer force.  I would call the work done at ask.sagemath.org and pretextbook.org actually very valuable in these terms, even if little of it goes directly toward developing new capabilities in Sage.  (CoCalc also performs a lot of this customer base development/service but since it is at least partly non-volunteer I put it in a different category.)

I think the easiest way out of this whole business actually is to have a relatively small bugfix 9.1 changeset, with OFFICIAL BINARIES in Python 2 posted, and those left up indefinitely, clearly marked as "Python 2".   (Having a choice of Sage cell server as well for some indeterminate period would be good, but again it is a lot to presume on Andrey alone for that.)  Does anyone know whether it is easy to use binary-pkg to create Python 2 binaries - it is as simple as adding a line in the sage.yaml file, or would one have to do something on the command line e.g. an env var first?

- kcrisman

[Andrey Novoseltsev]

On Monday, 6 January 2020 03:13:02 UTC-7, Frédéric Chapoton wrote:

the wiki promise was written by myself alone, and I think it should be changed. I think there will be very few (or  no) people that will desire to compile sage 9.0 with python 2. And 9.1 even more. People that want to be up to date do not care about py2.

I therefore strongly vote for the immediate drop of python2, and for removing the promise that 9.1 will be py2-compatible

I think that is the most important voice so far. When a single person on his own accord made a statement and posted it on a wiki page it should not be considered an official and unbreakable promise. When that same person wants to pull it back, and for a good reason, why should not we allow this?

We can talk about what would be nice, like running multiple cell servers, but let's pay attention to a) what is really happening and b) what are people actually willing to do in the future. I am personally not willing to work on multiple cell servers - those who really want and have machines available can quite easily set up their own server, I have tried hard to make it happen a long time ago. Others HAVE to switch to Python 3 sooner or later no matter what. Well, that time is now. I've timed the switch as best as I could releasing Python 3 version on Christmas and giving those preparing for the new term two weeks to adjust. Even with running another server some adjustments would be necessary.

Having a deprecation for print statement a year ago would be very useful and it was discussed and agreed on - as far as usefulness goes. But nobody actually worked on that change, which, I suspect, would be much easier than maintaining compatibility with two Pythons now.

I want to reiterate again that I am personally not affected by the decision made here in any way. But after years of adjusting SageMathCell to new versions of Sage, IPython, and now Python I think I have some idea of effort involved for those who do the porting. And I strongly believe that efforts of these people are too valuable to be spent on this support.

[Travis Scrimshaw]

I agree that the Python3 only version of Sage should be called 10.0 given the large backwards incompatible changes that result from the Python3 change. Furthermore, I also concur that we should release a version 9.1 (on an accelerated schedule) that is our official deprecation version telling people they will need to adapt their code in the next version.

Best,

Travis

[Eric Gourgoulhon]

Le lundi 6 janvier 2020 14:21:56 UTC+1, E. Madison Bray a écrit :

I agree with Nils.  There should be at least a one release deprecation
period.  Also, while I don't think we use any kind of real semantic
versioning, I think we should name a Python 3-only release 10.0 as
it's a very major backwards-incompatibility change.

On the other hand, for the end user the major backwards-incompatibility change already happened: a Python 2-only piece of code will break immediately in any Sage 9.0 binary. Can we really say to the end user: "to solve your issue, download SageMath sources and compile them with ./configure --with-python=2" ?

As for developers, the Python 3 switch has been discussed for something like 2 years, so what would be the point to extend that (effective) deprecation period? (maybe I am missing something here)

Eric.

[Nils Bruin]

On Monday, January 6, 2020 at 10:30:23 AM UTC-8, Eric Gourgoulhon wrote:

On the other hand, for the end user the major backwards-incompatibility change already happened: a Python 2-only piece of code will break immediately in any Sage 9.0 binary. Can we really say to the end user: "to solve your issue, download SageMath sources and compile them with ./configure --with-python=2" ?

That's a different issue. If we can't say that, the appropriate fix is to host py2 binaries as well (but hide them a little bit).

[rjf]

just curious when this ends.  Python 4 awaits.  

RJF

[William]

This is just some clarification and remarks related to what Karl wrote, and not really super relevant to this thread...

On Monday, January 6, 2020 at 7:17:16 AM UTC-8, kcrisman wrote:

Because a lot of Sage users are also developers (presumably a higher proportion than for Python or Mathematica!), a lot of Sage work has had developers in mind.  However, I think it is pretty important to pay attention to the users, the vast majority of whom we don't ever hear from, since they aren't paying for Sage.

As an example, if there were a coordinated strategy that a company would make, I can't imagine that there wouldn't have been two versions of Sage cell server going at once - or even an addition (temporary) to the drop-down menu saying "Sage with Python 2", for transitioning.

For CoCalc (run by a company), we have many different versions of Sage installed for projects to use.  For Jupyter  notebooks, users just explicitly select whatever version they want (via kernel selection), and it stays selected unless they change it.  For the sage command line, we have a "sage_select" command to set the default, or the user can type "sage-[version]" to get a specific version.

/ext/bin/sage_select <version>

Available versions: develop, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, 8.9, 9.0

For Sage worksheets, they just use whatever is "sage" in the PATH, which is a problem since there's a TON of python2 content used on cocalc, much made by paying customers (with thousands of students), so we can't break it.  We've been discussing how to move forward, slowly but surely, here https://github.com/sagemathinc/cocalc/issues/4212/. It's a confusing and tricky problem because Sage worksheets don't encode the version of Sage they use (dumb design decision by me) -- Jupyter is much better in this regard.   In any case, we don't have the luxury of break things for our paying customers.

 

I don't know if I want to ask the very capable, but very busy, Andrey N. to do that!  But once again, sage-devel will never know, because the users didn't pay anything,

Users (at least UTMOST) does sometimes pay a little bit to cover some hosting (thanks NSF!), but it doesn't cover what Andrey does, and I fully support his decision.  I wish I had the money to pay to support maintenance of other versions of the Sage cell server, but I don't.

 

William

[E. Madison Bray]

On Mon, Jan 6, 2020 at 7:23 PM Travis Scrimshaw <tsc...@ucdavis.edu> wrote:
>
> I agree that the Python3 only version of Sage should be called 10.0 given the large backwards incompatible changes that result from the Python3 change. Furthermore, I also concur that we should release a version 9.1 (on an accelerated schedule) that is our official deprecation version telling people they will need to adapt their code in the next version.

This would be the right thing to do from a software maintenance
standpoint and from a user-facing standpoint. The vast majority of
Sage users are NOT developers and this community has too much a habit
in general of moving fast and breaking things for the short-term
benefit of power-users/developers (who in general are more able to run
their own builds as needed).

As it is I think we rushed too fast into a Python 3 default release
but in that case I think it was necessary and unavoidable. Now that
that's out (grace à Frédéric) we can take a minute to reflect on how
that transition goes for users and make the first Python 3-only
release even stronger.

[E. Madison Bray]

For this very reason I think there ought to be Python 2 binary
releases for 9.0 as well. I'm building both for Windows, but I'm not
in control of the others.

[E. Madison Bray]

On Tue, Jan 7, 2020 at 12:06 AM rjf <fat...@gmail.com> wrote:
>
> just curious when this ends. Python 4 awaits.

It's already ended. Python 4 is not going to be the major
backwards-compatibility breaker that Python 3 was. It's just going to
be the next release after 3.9, at which point we can also hopefully
finally stop talking about "Python 3" and just call it "Python" again.
This is similar to how Jinja2 is called Jinja2 because it was
completely incompatible with Jinja 1.x and so needed a "different
name" as it were. But now that Jinja2 is long-since the de facto
Jinja, Jinja 3.0 will just be referred to as "Jinja" again; no more
Jinja2.

In the more extreme end of things Perl 6 has finally stopped being
"Perl" altogether and is called Raku now. Fortunately even Python 3
is not as big a "disaster" from a backwards-incompatibility
perspective that it hasn't necessitated a completely new name for the
language; the "Python 3" distinction has only been relevant in the
context of porting from Python 2 and now that that's mostly done in
most of the community we can finally start to drop it soon...

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/c372b39f-6232-48a5-bc98-e452c905ab1c%40googlegroups.com.

[kcrisman]

William, thanks for this relevant (if not directly, as you say) point of view.

For CoCalc (run by a company), we have many different versions of Sage installed for projects to use.  For Jupyter  notebooks, users just explicitly select whatever version they want (via kernel selection), and it stays selected unless they change it.  For the sage command line, we have a "sage_select" command to set the default, or the user can type "sage-[version]" to get a specific version.

/ext/bin/sage_select <version>

Available versions: develop, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, 8.9, 9.0

For Sage worksheets, they just use whatever is "sage" in the PATH, which is a problem since there's a TON of python2 content used on cocalc, much made by paying customers (with thousands of students), so we can't break it.  We've been discussing how to move forward, slowly but surely, here https://github.com/sagemathinc/cocalc/issues/4212/. It's a confusing and tricky problem because Sage worksheets don't encode the version of Sage they use (dumb design decision by me) -- Jupyter is much better in this regard.   In any case, we don't have the luxury of break things for our paying customers.

Exactly.  By the way, even 8.2 - impressive!

 

 

I don't know if I want to ask the very capable, but very busy, Andrey N. to do that!  But once again, sage-devel will never know, because the users didn't pay anything,

Users (at least UTMOST) does sometimes pay a little bit to cover some hosting (thanks NSF!), but it doesn't cover what Andrey does, and I fully support his decision. 

Of course, and I hope that was clear in what I said.

[Eric Gourgoulhon]

One may argue that there are already available Python 2 binaries: the 8.9 binaries.

Is it worth to spend time and energy to build new Python 2 binaries?

Wouldn't a message like "if you insist in running Python 2-only code, please use the 8.9 binaries" be sufficient?

IMHO, most end users should now that Python 2 is dead (the younger ones even do not know that such a thing had existed) and that "print bla" should be changed to "print(bla)".

Eric.

[Dima Pasechnik]

Why 8.9? One can build 9.0 with python2 just fine.

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/98c43f60-8613-4a09-b244-51b11a85cfd4%40googlegroups.com.

[Eric Gourgoulhon]

Le mardi 7 janvier 2020 14:35:45 UTC+1, Dima Pasechnik a écrit :

Why 8.9? One can build 9.0 with python2 just fine.

Yes I know, but I was speaking about the time and energy to actually build those and distribute them. IMHO, there are more pressing issues, like the handling of ipython7 and Python 3.8 mentioned in Frédéric's original message.

That being said, I am not involved in the preparation of Sage binaries and your voice or that of Erik is more important than mine.

Eric.

[E. Madison Bray]

On Tue, Jan 7, 2020 at 2:16 PM Eric Gourgoulhon <egourg...@gmail.com> wrote:
>
> Le mardi 7 janvier 2020 13:25:04 UTC+1, E. Madison Bray a écrit :
>>
>> On Mon, Jan 6, 2020 at 7:30 PM Eric Gourgoulhon <egourg...@gmail.com> wrote:
>> >
>> > On the other hand, for the end user the major backwards-incompatibility change already happened: a Python 2-only piece of code will break immediately in any Sage 9.0 binary. Can we really say to the end user: "to solve your issue, download SageMath sources and compile them with ./configure --with-python=2" ?
>> > As for developers, the Python 3 switch has been discussed for something like 2 years, so what would be the point to extend that (effective) deprecation period? (maybe I am missing something here)
>>
>> For this very reason I think there ought to be Python 2 binary
>> releases for 9.0 as well. I'm building both for Windows, but I'm not
>> in control of the others.
>
>
> One may argue that there are already available Python 2 binaries: the 8.9 binaries.
> Is it worth to spend time and energy to build new Python 2 binaries?
> Wouldn't a message like "if you insist in running Python 2-only code, please use the 8.9 binaries" be sufficient?

If that were the case, that should have been communicated clearly
during the 8.9 release, and it wasn't (as it is we don't communicate
changes between versions clearly enough, but that would be a major
omission).

> IMHO, most end users should now that Python 2 is dead (the younger ones even do not know that such a thing had existed) and that "print bla" should be changed to "print(bla)".

Most users I've encountered (especially new users) don't even realize
Sage is written in Python, and don't know anything about Python 2 vs
Python 3. Fortunately, as you say, for Sage the most major
user-visible change is just print statements and we've already been
warning about that for a while. But existing users' code can break in
other new an exciting ways (e.g. map).

[Dima Pasechnik]

On Tue, 7 Jan 2020, 06:04 William, <wst...@gmail.com> wrote:

This is just some clarification and remarks related to what Karl wrote, and not really super relevant to this thread...

On Monday, January 6, 2020 at 7:17:16 AM UTC-8, kcrisman wrote:

Because a lot of Sage users are also developers (presumably a higher proportion than for Python or Mathematica!), a lot of Sage work has had developers in mind.  However, I think it is pretty important to pay attention to the users, the vast majority of whom we don't ever hear from, since they aren't paying for Sage.

As an example, if there were a coordinated strategy that a company would make, I can't imagine that there wouldn't have been two versions of Sage cell server going at once - or even an addition (temporary) to the drop-down menu saying "Sage with Python 2", for transitioning.

For CoCalc (run by a company), we have many different versions of Sage installed for projects to use.  For Jupyter  notebooks, users just explicitly select whatever version they want (via kernel selection), and it stays selected unless they change it.  For the sage command line, we have a "sage_select" command to set the default, or the user can type "sage-[version]" to get a specific version.

/ext/bin/sage_select <version>

Available versions: develop, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, 8.9, 9.0

you could add something like 9.0_py2, as 9.0 is still meant to be fully functional with Python 2.

For Sage worksheets, they just use whatever is "sage" in the PATH, which is a problem since there's a TON of python2 content used on cocalc, much made by paying customers (with thousands of students), so we can't break it.  We've been discussing how to move forward, slowly but surely, here https://github.com/sagemathinc/cocalc/issues/4212/. It's a confusing and tricky problem because Sage worksheets don't encode the version of Sage they use (dumb design decision by me) -- Jupyter is much better in this regard.   In any case, we don't have the luxury of break things for our paying customers.

 

I don't know if I want to ask the very capable, but very busy, Andrey N. to do that!  But once again, sage-devel will never know, because the users didn't pay anything,

Users (at least UTMOST) does sometimes pay a little bit to cover some hosting (thanks NSF!), but it doesn't cover what Andrey does, and I fully support his decision.  I wish I had the money to pay to support maintenance of other versions of the Sage cell server, but I don't.

 

William

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/99d1c833-d393-46b0-a7d3-72e184ba047b%40googlegroups.com.

[Jonathan Kliem]

Just a silly question. How do we know that the code remains python2 compatible in the first place?

As far as I can see all the patchbots run python3 so it is very easy to break something. 9.0 is supposed to be fully python2 compatible, but are the doctests being tested for that?

If we don't test the upcoming tickets for python2 compatibility, we have de facto stopped supporting python2 already.

Jonathan

[kcrisman]

On Tuesday, January 7, 2020 at 2:17:54 PM UTC-5, Jonathan Kliem wrote:

Just a silly question. How do we know that the code remains python2 compatible in the first place?

As far as I can see all the patchbots run python3 so it is very easy to break something. 9.0 is supposed to be fully python2 compatible, but are the doctests being tested for that?

I think on sage-release at least a few people are testing for this, or have been, at any rate.

 

If we don't test the upcoming tickets for python2 compatibility, we have de facto stopped supporting python2 already.

If we have a few people run with the python 2 config for 9.1 I think that is reasonable.  We don't claim the betas are py2 compatible. 

But yes, if we were to support both long-term (not that we are), one would want that to be the case.

I'll once again ask if someone knows how I might (easily) make a py2 binary for Sage 9.0 using binary-pkg.  I don't think I can necessarily use the configure command before running that since everything happens "inside" of binary-pkg.

Thanks,

- kcrisman

[Sébastien Labbé]

The question I have is the folllowing. Suppose a user have 10k lines of SageMath code (notebooks, files, etc.) working well in 8.9 which obviously will get broken in the default 9.0 mostly because of:

 - print

 - comparisons

 - iterkeys, iteritems

 - and few other particular changes

1. What is the best workflow for such users to adapt to the 9.0, 9.1, etc. series?

2. How can the version 9.1 of SageMath be made so that it is really helpful? 

3. How can a version 9.1 or 9.0 running Python 2 can be more helpful than the version 8.9 running Python 2? (Let's recall that the old code was running in 8.9 so it does not use any of the new features added in 9.0)

Sébastien

[Dima Pasechnik]

On Tue, Jan 7, 2020 at 9:45 PM Sébastien Labbé <sla...@gmail.com> wrote:
>
> The question I have is the folllowing. Suppose a user have 10k lines of SageMath code (notebooks, files, etc.) working well in 8.9 which obviously will get broken in the default 9.0 mostly because of:

notebooks as in sagenb?
This would really be not fun at all, as sagenb does not run on py3.

> - print
> - comparisons
> - iterkeys, iteritems
> - and few other particular changes
>
> 1. What is the best workflow for such users to adapt to the 9.0, 9.1, etc. series?
>
> 2. How can the version 9.1 of SageMath be made so that it is really helpful?
>
> 3. How can a version 9.1 or 9.0 running Python 2 can be more helpful than the version 8.9 running Python 2? (Let's recall that the old code was running in 8.9 so it does not use any of the new features added in 9.0)

We did see one-liners working in 8.9 and broken in 9.0, to do with graphics.
Every new release not only adds features, it fixed bugs (and
introduces new ones, obviously :-))
Thus, the closer you are to the current release, the better.

>
> Sébastien

>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/bcd007fd-7b22-4d5d-9aee-44005ebd684b%40googlegroups.com.

[Volker Braun]

* I think its not too difficult to write code that is Python 2.7 + 3.x (for high enough x) compatible, so its not a super pressing issue

* We do have a Python 2 buildbot to test for regressions

* For semver reasons we should drop Python 2.7 support in Sage 10, not 9.1

Having said that, I'm fine with an accelerated Sage 9 -> Sage 10 schedule, maybe a month or two instead of the usual 3-4. Though first we should take the opportunity and see if there are any outstanding Python 3 bugs now that we have more data. For example it would be nice if a build with SAGE_DEBUG=yes would pass tests. There are a few more regressions, e.g. #28817

[E. Madison Bray]

Accelerated is fine, but I should add that fully *removing* Python 2
support is not a trivial task either (it's mostly mechanical work, but
there's a LOT of it). https://trac.sagemath.org/ticket/28000 has a
few steps towards it but is far from complete.

Fully removing Python 2-isms from the codebase is not necessarily a
prerequisite to dropping Python 2 support, but I would be hesitant to
leave a monstrous partially-hybrid codebase around for too long.

[Timo Kaufmann]

I have said this before, but I feel like the point was dropped out of the discussion so I'll stress it again. The major issue here is *not* the compatibility of sage's own codebase. A few "from __future__ import"'s are not so bad.

The issue is that python2 compatibility forces us to use outdated versions of a lot of libraries, since many libraries have dropped python2 support a while ago. This is a big headache especially for packagers. Those outdated libraries are generally not available on distros. At the same time sage is usually not compatible with the newer versions. Sage is already difficult to package, and that makes it a lot more difficult.

Case in point, my own efforts to update sage on nixos to 9.0 and python3 have been blocked by incompatibilities in a new sphinx version (https://trac.sagemath.org/ticket/28856). I have already put some hours into this, but I simply don't have time to maintain a sage package if all the compatibility efforts fall to packagers. Another example is Antonio's multiple thousand line patch he mentioned earlier.

The alternatives are:

(1) create infrastructure in sage that allows us to use newer dependencies in python3 (https://trac.sagemath.org/ticket/28190), and make sage compatible with both the py2 and py3 version. This is a lot of effort.

(2) drop python2 support ASAP, e.g. in 9.1, and make sage compatible with the newer libraries

(3) continue as-is and leave it to packers to "deal with it"

As I said, (1) is a lot of effort. It would be the nicest solution, but I don't think there are enough people willing to do the work quickly enough. Especially if that effort will be wasted in a couple of months' time once py2 support is dropped.

(2) is my favorite solution. It re-synchronizes upstream with downstream distributions (which will most likely no longer package the python2 version anyways). I see very little downsides to this. Yes, some people will need longer to switch over their codebases. But their existing codebases, having been written before 9.1, will work with sage 9.0. So why not continue using it as long as the porting efforts take?

And again, I think (3) is a very bad solution. Sage already takes many times the packaging effort than most other packages. If this effort increases, I won't be willing or able to keep it up.

Sorry for the rant, I hope the email didn't get too long for people to read.

[E. Madison Bray]

On Fri, Jan 10, 2020 at 10:41 AM Timo Kaufmann <eisf...@gmail.com> wrote:
>
> I have said this before, but I feel like the point was dropped out of the discussion so I'll stress it again. The major issue here is *not* the compatibility of sage's own codebase. A few "from __future__ import"'s are not so bad.
>
> The issue is that python2 compatibility forces us to use outdated versions of a lot of libraries, since many libraries have dropped python2 support a while ago. This is a big headache especially for packagers. Those outdated libraries are generally not available on distros. At the same time sage is usually not compatible with the newer versions. Sage is already difficult to package, and that makes it a lot more difficult.

Can you be more specific about this? What is it about Sage's upstream
codebase maintaining backwards-compatibility for Python 2 that
prevents you from packaging it for Python 3 only, given that it does
support Python 3? No one is saying that just because upstream support
is maintained for Python 2 for one or two (at the most) more releases,
any downstream packagers have to package it for Python 2.

[François Bissey]

We don’t. I only support python3 anymore on gentoo - because support for python2
ipython/jupyter as been removed from the main tree.
I am the lucky one because I still have an ipython-5 I can pull as a dependency,
most of my colleagues have to go ipython-7+ which is python3 only and requires
heavy patching. Some interesting packages we may want to use that are python3 only
and that will make our life difficult because we have to move forward regardless
* ipython-7
* matplotlib-3
* sphinx-2
* numpy-1.17
* scipy-1.3+

The main person that has valid reason to want longer support for python2 is William.
As he mentions, he has paying customers.

At this point my only reasonable suggestion is to have a python2 compatibility branch
for a while as sage-9.x while some of us plow on ahead without in sage-10.x.
That’s unreasonable on the release manager unless someone step up to help.

François

[Vincent Delecroix]

Le 06/01/2020 à 00:24, Nils Bruin a écrit :
> On Sunday, January 5, 2020 at 2:51:44 PM UTC-8, Nils Bruin wrote:
>>
>> I don't think there's a doubt that we'll drop py2.7 support. This Wiki,
>> however, has been referred to in many places describing the py2/py3
>> transition.
>>
>
> One of these places is the FAQ, which is official documentation:
>
> http://doc.sagemath.org/html/en/faq/faq-usage.html#can-i-use-sagemath-with-python-3-x
>
> The 9.0 FAQ still mentions:
>
> As of August 2019, most of SageMath works fine with Python 3. However, we
> still consider Python 3 support to be experimental and no official Python 3
> release has been made yet.
>
> That would probably be good to adjust too: there is now an official Py3
> release and I don't think we consider it experimental anymore -- unless
> sage in its entirety is still considered experimental :-).
>

See https://trac.sagemath.org/ticket/28980 (even though Frédéric claims
that it might have been done in another ticket).

[Isuru Fernando]

I'll give an example. sage has rpy2 2.8.2.

rpy2 latest is 3.2.4 but 3.x series is python 3 only and sage requires breaking changes to support rpy2 3.x. See https://trac.sagemath.org/ticket/28494#comment:6

While python2 support for rpy2 is required, sage codebase can't be updated to 3.x (unless someone adds code to detect rpy2 version and support both 2.8.2 and 3.2.4)

 

Isuru

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34ZqHf%2BrzoSD8juhhk%3D4YF1WB6BHZuxAOO0ZQYk6KhhX_g%40mail.gmail.com.

[E. Madison Bray]

On Fri, Jan 10, 2020 at 11:49 AM Isuru Fernando <isu...@gmail.com> wrote:
>
>
> On Fri, Jan 10, 2020 at 3:53 PM E. Madison Bray <erik....@gmail.com> wrote:
>>
>> On Fri, Jan 10, 2020 at 10:41 AM Timo Kaufmann <eisf...@gmail.com> wrote:
>> >
>> > I have said this before, but I feel like the point was dropped out of the discussion so I'll stress it again. The major issue here is *not* the compatibility of sage's own codebase. A few "from __future__ import"'s are not so bad.
>> >
>> > The issue is that python2 compatibility forces us to use outdated versions of a lot of libraries, since many libraries have dropped python2 support a while ago. This is a big headache especially for packagers. Those outdated libraries are generally not available on distros. At the same time sage is usually not compatible with the newer versions. Sage is already difficult to package, and that makes it a lot more difficult.
>>
>> Can you be more specific about this? What is it about Sage's upstream
>> codebase maintaining backwards-compatibility for Python 2 that
>> prevents you from packaging it for Python 3 only, given that it does
>> support Python 3? No one is saying that just because upstream support
>> is maintained for Python 2 for one or two (at the most) more releases,
>> any downstream packagers have to package it for Python 2.
>
>
> I'll give an example. sage has rpy2 2.8.2.
> rpy2 latest is 3.2.4 but 3.x series is python 3 only and sage requires breaking changes to support rpy2 3.x. See https://trac.sagemath.org/ticket/28494#comment:6
> While python2 support for rpy2 is required, sage codebase can't be updated to 3.x (unless someone adds code to detect rpy2 version and support both 2.8.2 and 3.2.4)

That seems like the obvious approach to me. As it is I've long felt
that Sage should be more flexible in its dependencies where
possible/necessary. With most Python packages it's easy as most have
a <package>.__version__ and its not so hard to define some variable
like IS_RPY_2 and just have two separate branches. I have things like
that all over the place in other packages to support e.g. different
Numpy versions or work around version-specific bugs.

[Vincent Delecroix]

+1. I also do that with many Python packages that I develop and depend
on Sage. This is the only way I found to support multiple Sage versions.

[William Stein]

The main person that has valid reason to want longer support for python2 is William.
As he mentions, he has paying customers.

I am 100% satisfied for my use case with cocalc by just keeping a copy of sage-8.9 available longterm.   Fortunately, I don't need any sage-9.x versions to support python2.

At this point my only reasonable suggestion is to have a python2 compatibility branch
for a while as sage-9.x while some of us plow on ahead without in sage-10.x.
That’s unreasonable on the release manager unless someone step up to help.

François

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/vYlbnAwKATM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/BBC8CF32-2E9D-4DC0-BBE5-5130E15B3B14%40gmail.com.

[Nathan Dunfield]

On Friday, January 10, 2020 at 8:11:43 AM UTC-8, William wrote:

The main person that has valid reason to want longer support for python2 is William.
As he mentions, he has paying customers.

I am 100% satisfied for my use case with cocalc by just keeping a copy of sage-8.9 available longterm.   Fortunately, I don't need any sage-9.x versions to support python2.

I think that for nearly all users, in practice sage < 9.0 will mean Python 2 and sage >= 9.0 will mean Python 3.  There will be a handful of folks who use (sage-9.x + Python 2) for whatever few values of x this will work for, but I agree with William and others that in nearly all cases just using 8.9 would suffice.

Given that the period of official dual Python 2/Python 3 support is going to be a few months, not years, I see no harm in removing Python 2 support in Sage 9.1.

In terms of helping users with thousands of lines of Python 2 based Sage code, trying to keep building Sage 8.9-the-distribution binaries for future OS releases as they come out is probably a lot more effective than working hard to push back the date that Python 2 support is dropped from Sage 9.x.

Best,

Nathan

[Antonio Rojas]

El viernes, 10 de enero de 2020, 14:54:24 (UTC+1), E. Madison Bray escribió:

That seems like the obvious approach to me.  As it is I've long felt
that Sage should be more flexible in its dependencies where
possible/necessary.  With most Python packages it's easy as most have
a <package>.__version__ and its not so hard to define some variable
like IS_RPY_2 and just have two separate branches.  I have things like
that all over the place in other packages to support e.g. different
Numpy versions or work around version-specific bugs.

I've opened https://trac.sagemath.org/ticket/28988 for rpy. But at this point the major issues are python 3.8 and ipython 7, and I don't see how one could support several versions of them without forking hundreds of doctests. Both updates require multi-thousand-lines patches, due to changes in dict sorting and hashes.

[E. Madison Bray]

That remains a fault of over-reliance on doctests. I don't think
downstream packaging is a good enough reason to push sage to rush
things in such a way that is not well-communicated to the user
community. If you need to have a multi-thousand-line patch then so be
it. A patch is a patch.

[Timo Kaufmann]

Am Montag, 13. Januar 2020 17:33:56 UTC+1 schrieb E. Madison Bray:

On Sat, Jan 11, 2020 at 9:24 AM Antonio Rojas <nqn...@gmail.com> wrote:
>
> El viernes, 10 de enero de 2020, 14:54:24 (UTC+1), E. Madison Bray escribió:
>>
>> That seems like the obvious approach to me.  As it is I've long felt
>> that Sage should be more flexible in its dependencies where
>> possible/necessary.  With most Python packages it's easy as most have
>> a <package>.__version__ and its not so hard to define some variable
>> like IS_RPY_2 and just have two separate branches.  I have things like
>> that all over the place in other packages to support e.g. different
>> Numpy versions or work around version-specific bugs.
>
>
> I've opened https://trac.sagemath.org/ticket/28988 for rpy. But at this point the major issues are python 3.8 and ipython 7, and I don't see how one could support several versions of them without forking hundreds of doctests. Both updates require multi-thousand-lines patches, due to changes in dict sorting and hashes.

That remains a fault of over-reliance on doctests.

What else should we rely on? Also, doctests are not the only things that fail with current python3 libraries.

I don't think
downstream packaging is a good enough reason to push sage to rush
things in such a way that is not well-communicated to the user
community.  If you need to have a multi-thousand-line patch then so be
it.  A patch is a patch.

That is unfortunate. I agree that "a patch is a patch", but in my view the conclusion should be the opposite: Upstream should strive for no patches to be necessary (except maybe to work around very distro-specific quirks). No 5000 line patches and no 5 line patches.

For me this decision means that sage on nixos will likely be stuck on python2 for a while, and I can only hope that the python2 infrastructure keeps working for long enough.

I still don't understand the reasoning here. From my point of view (which is of course biased), keeping python2 compatibility has a huge downside. At the same time, keeping it has very little upside. The main reasoning seems to be to give users time to update their code. But the value-addition of keeping python2 in newer sage versions vs just telling users to use 8.9 / 9.0 with python2 seems very small to me. On the contrary, newer features being python3 exclusive might give a good incentive for updating. Otherwise we just postpone the problem until we finally do drop support.

[Dima Pasechnik]

On Sun, Jan 5, 2020 at 7:44 PM Frédéric Chapoton <fchap...@gmail.com> wrote:
>
> Hello,
>
> I would like to suggest that the sooner we drop Python 2 support the better. We still need to handle the upgrade to ipython7 and the compatibility with python 3.8. All this will be made very difficult if we insist on maintaining a codebase that is both compatible with python 2 and python 3.
>
> So, please vote :
>
> Do you agree that sage release 9.1 (and most of the 9.1.betas) will not be kept compatible with Python 2 ?

As far as I am concerned, the sooner py2 is dropped on the "main"
branches, the better.

This does not preclude making separate maintenance releases for py2, if need be.



>
> Frédéric
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/692fc332-d9e6-4e6a-80fe-2e89cf0b488f%40googlegroups.com.

[Frédéric Chapoton]

This has been a long discussion already. Let me try to summarize. My question was :

  Do you agree that sage release 9.1 (and most of the 9.1.betas) will not be kept compatible with Python 2 ?

Some people give a more or less clear positive answer :

Chapoton, Bissey, Gourgoulhon, Kaufmann, Novoseltsev, Koeppe, Mantysalo, Rojas, Isuru, Stein, Dunfield, Pasechnik (12)

Some people give a strong negative answer :

Bruin, Bray (2)

Some people did not answer the question in a clear way, but still said something :

Palmieri, Roe, King, Crisman, Delecroix, Scrimshaw, Kliem, Labbé, Braun (9)

Among them, I would say that maybe 5 people were rather negative on the question.

I hope that I have not forgotten anybody or misinterpreted wildly.

[A] It seems to me that the conclusion of the vote is rather clear, and that we should not feel obliged to make 9.1 compilable with python2.

==> I have therefore changed the controversial wiki page, to make only the statement that 9.0 is compilable with python2. No more promise for the future.

[B] I understand and approve the concerns about our user base. We should really try harder to listen more and help people that are simple users.

So here is my proposal.

* Starting from now, we allow ourselves to move on, using 9.1 betas and further releases for external python3 updates, including switch to ipython7, which seems to me the most urgent matter. But we also do not introduce python3-only code in our own code base if we can avoid it.

* Concerning semantic versioning, I would advocate to use 10.0 as the first release where all compatibility code will be removed, hence pure py3. This may take some time, given the sparse work force. Once done, we allow ourself to use python3-only syntax in our own codebase, such as "yield from".

Therefore, we can use 9.1, 9.2 and so on until somebody™ does the job of cleaning up sagemath codebase from remains of py2-only code.

I have tried to make a balanced and reasonable proposal. Applause or scream at your will.

Frederic

Le dimanche 5 janvier 2020 20:44:10 UTC+1, Frédéric Chapoton a écrit :

Hello,

I would like to suggest that the sooner we drop Python 2 support the better. We still need to handle the upgrade to ipython7 and the compatibility with python 3.8. All this will be made very difficult if we insist on maintaining a codebase that is both compatible with python 2 and python 3.

So, please vote :

Frédéric

[William Stein]

Strong *applause* from me.  (This is a very hard problem and I continue to be amazed and greatly appreciate what everybody has done related to Python3 support in Sage!)

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/vYlbnAwKATM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/9db91206-3a7c-434a-8b07-7656e5b5f458%40googlegroups.com.

[Nils Bruin]

On Tuesday, January 14, 2020 at 9:01:42 AM UTC-8, Frédéric Chapoton wrote:

So here is my proposal.

* Starting from now, we allow ourselves to move on, using 9.1 betas and further releases for external python3 updates, including switch to ipython7, which seems to me the most urgent matter. But we also do not introduce python3-only code in our own code base if we can avoid it.

I think for development and stability, we should take heed of the fact that trac has some recent tickets addressing issues with py3 that are only being uncovered now. The reality is that prior to 9.0, sage/py3 got only very limited testing outside doctests and developers working on it. Doctests by no means cover the full functionality of sage, and I would expect that in the next few months a lot more issues will be found. I'd be a little hesitant dropping py2 support under those conditions: being able to drop back on py2 in these situations for a while seems like a decent mitigation strategy, and would probably help with debugging.

I think similar thinking led to the original proposed and published strategy on the wiki, and it still makes sense. In practice, sage/py3 is not really a proven product yet; just because people won't start really using it until there's actually an official release with it.

 I appreciate the good intentions of "not introducing py3-only code in our own code-base if we can avoid it", but how much does that help? If there's one such change in a file that gets loaded into sage upon start-up, or which needs to be parsed during build, the possibility of running sage on py2 is gone, isn't it?

I agree it's perhaps frustrating that the process of moving sage from py2 to py3 hasn't completed with py2 falling out of support, but having a first official release based on py3, while being the biggest step (and a big applause for everybody who contributed to this herculean task!) is unfortunately not the completion. The bug reporting and fixing that follows it shouldn't just be ignored.

[Michael Orlitzky]

On 1/14/20 12:01 PM, Frédéric Chapoton wrote:
>
> [A] It seems to me that the conclusion of the vote is rather clear, and
> that we should not feel obliged to make 9.1 compilable with python2.
>

Since I'm not on that list yet, one thing is for sure: there are going
to be two consecutive versions of sage, one where python-2.x works, and
one where it doesn't. Users won't care too much which versions those are
unless there's a feature or bugfix that they need in the new version --
they can just use the old one.

Most projects solve this with multiple release branches. When you break
backwards-compatibility in a major way, you keep the previous version
(before the break) in a separate branch, and backport critical fixes to
it for a limited time. It's not how the sage release process works, but
I don't see why we couldn't do it this once, to keep everyone happy:

1. Branch off the 9.0 release
2. Drop compatibility in 9.1
3. Proceed as normal
4. If there are any major bugs found in 9.1, backport the fix to
the 9.0 branch, and release a 9.0.1 that supports python-2.x.

Mathematical bug fixes that merge clean and pass the tests could also
make the cut, but ideally step (4) would require little work. This would
buy users a few more months of python2 compatibility, and lets the rest
of us move on.

The main downside is that this relies solely on the release manager to
merge bugfix tickets twice, into both release branches. So if he thinks
this is a waste of time (understandable), then it's a non-starter.

[Matthias Koeppe]

On Tuesday, January 14, 2020 at 8:01:18 PM UTC-5, Michael Orlitzky wrote:

  1. Branch off the 9.0 release
  2. Drop compatibility in 9.1
  3. Proceed as normal
  4. If there are any major bugs found in 9.1, backport the fix to
     the 9.0 branch, and release a 9.0.1 that supports python-2.x.

Mathematical bug fixes that merge clean and pass the tests could also
make the cut, but ideally step (4) would require little work. This would
buy users a few more months of python2 compatibility, and lets the rest
of us move on.

I agree that there needs to be a 9.0 maintenance branch – if only to be the designated upstream for distributors that want to keep maintaining a py2 version of sage. 

 

The main downside is that this relies solely on the release manager to
merge bugfix tickets twice, into both release branches. So if he thinks
this is a waste of time (understandable), then it's a non-starter.

The maintenance branch could also be maintained by another developer. 

[Matthias Koeppe]

On Tuesday, January 14, 2020 at 12:01:42 PM UTC-5, Frédéric Chapoton wrote:

So here is my proposal.

* Starting from now, we allow ourselves to move on, using 9.1 betas and further releases for external python3 updates, including switch to ipython7, which seems to me the most urgent matter. But we also do not introduce python3-only code in our own code base if we can avoid it.

* Concerning semantic versioning, I would advocate to use 10.0 as the first release where all compatibility code will be removed, hence pure py3. This may take some time, given the sparse work force. Once done, we allow ourself to use python3-only syntax in our own codebase, such as "yield from".

Therefore, we can use 9.1, 9.2 and so on until somebody™ does the job of cleaning up sagemath codebase from remains of py2-only code.

I have tried to make a balanced and reasonable proposal. Applause or scream at your will.

I agree with all of this.  

[Sébastien Labbé]

On Tuesday, January 14, 2020 at 6:01:42 PM UTC+1, Frédéric Chapoton wrote:

This has been a long discussion already. Let me try to summarize. My question was :

  Do you agree that sage release 9.1 (and most of the 9.1.betas) will not be kept compatible with Python 2 ?

here is my vote:

+1

 

[E. Madison Bray]

On Tue, Jan 14, 2020 at 10:41 AM Timo Kaufmann <eisf...@gmail.com> wrote:
>
> Am Montag, 13. Januar 2020 17:33:56 UTC+1 schrieb E. Madison Bray:
>>
>> On Sat, Jan 11, 2020 at 9:24 AM Antonio Rojas <nqn...@gmail.com> wrote:
>> >
>> > El viernes, 10 de enero de 2020, 14:54:24 (UTC+1), E. Madison Bray escribió:
>> >>
>> >> That seems like the obvious approach to me. As it is I've long felt
>> >> that Sage should be more flexible in its dependencies where
>> >> possible/necessary. With most Python packages it's easy as most have
>> >> a <package>.__version__ and its not so hard to define some variable
>> >> like IS_RPY_2 and just have two separate branches. I have things like
>> >> that all over the place in other packages to support e.g. different
>> >> Numpy versions or work around version-specific bugs.
>> >
>> >
>> > I've opened https://trac.sagemath.org/ticket/28988 for rpy. But at this point the major issues are python 3.8 and ipython 7, and I don't see how one could support several versions of them without forking hundreds of doctests. Both updates require multi-thousand-lines patches, due to changes in dict sorting and hashes.
>>
>> That remains a fault of over-reliance on doctests.
>
>
> What else should we rely on? Also, doctests are not the only things that fail with current python3 libraries.

Normal unit tests. The annoying thing about Sage's doctest-centric
testing framework (which is otherwise good) is that it tends to lead
people to thinking they can't write unit tests. But in fact there are
unit tests in Sage, and they're just invoked through the doctest
runner by writing "doctests" that run a unit test suite.

>> I don't think
>> downstream packaging is a good enough reason to push sage to rush
>> things in such a way that is not well-communicated to the user
>> community. If you need to have a multi-thousand-line patch then so be
>> it. A patch is a patch.
>
>
> That is unfortunate. I agree that "a patch is a patch", but in my view the conclusion should be the opposite: Upstream should strive for no patches to be necessary (except maybe to work around very distro-specific quirks). No 5000 line patches and no 5 line patches.
>
> For me this decision means that sage on nixos will likely be stuck on python2 for a while, and I can only hope that the python2 infrastructure keeps working for long enough.

Well, all the more reason to maintain Python 2 support a little longer
than to rush and break things.

[E. Madison Bray]

On Tue, Jan 14, 2020 at 11:49 AM Dima Pasechnik <dim...@gmail.com> wrote:
>
> On Sun, Jan 5, 2020 at 7:44 PM Frédéric Chapoton <fchap...@gmail.com> wrote:
> >
> > Hello,
> >
> > I would like to suggest that the sooner we drop Python 2 support the better. We still need to handle the upgrade to ipython7 and the compatibility with python 3.8. All this will be made very difficult if we insist on maintaining a codebase that is both compatible with python 2 and python 3.
> >
> > So, please vote :
> >
> > Do you agree that sage release 9.1 (and most of the 9.1.betas) will not be kept compatible with Python 2 ?
>
> As far as I am concerned, the sooner py2 is dropped on the "main"
> branches, the better.
>
> This does not preclude making separate maintenance releases for py2, if need be.

This I would be okay with, and I have always said we should have
maintenance branches, but the release manager doesn't want to do that
so *shrug*

[E. Madison Bray]

^ This says it better than I could.

[E. Madison Bray]

On Wed, Jan 15, 2020 at 2:37 AM Matthias Koeppe
<matthia...@gmail.com> wrote:
>
> On Tuesday, January 14, 2020 at 12:01:42 PM UTC-5, Frédéric Chapoton wrote:
>>
>> So here is my proposal.
>>
>> * Starting from now, we allow ourselves to move on, using 9.1 betas and further releases for external python3 updates, including switch to ipython7, which seems to me the most urgent matter. But we also do not introduce python3-only code in our own code base if we can avoid it.

I'm not exactly sure what you mean by this, but it might be agreeable
to me: I'm completely okay with adding Python 3 only dependencies and
even *code/features* so long as it's done without breaking
backwards-compatibility. When it comes to sage-the-distribution
(which I sense is where most of this friction is coming from, and yet
another reason to better separate sage from the
sage-the-distribution), if there are dependencies you want to update
that are Python 3-only I'd say go for it, but make it a separate SPKG
so that previous versions of the dependency can still work on Python 2
builds.

[Timo Kaufmann]

As others have said already, supporting both python2 and python3 dependencies is much more work than only supporting python2 or only supporting python3 versions.

I think dropping pyhon2 support with the next release remains the best option. Even better of course would be to keep the option for bugfix releases for 8.9 / 9.0. That would be a nice thing to have in general, but we could also just make an exception for this one release.

[Emmanuel Charpentier]

I second that. Unless someone presents us with a pressing case of a large Python 2-based library unportable to Python 3 (and therefore already in a dead-end), our limited workforce is better employed to maintain our Python 3 Sage.

In other words, "if you need a Python 2-specific fix, speak *now* or forever hold your peace..."

Corollary : we should update the places where we published our plans for Python 2 compatibility and point to the present thread, which can be summarized by "The Sagemath developers have decided not to maintain Python 2 compatibility beyond Sagemath 9.0." as an acceptable first-order approximation.

I agree that a Python 2-compatible maintenance branch *would* be a Good Thing (TM) to have. But I doubt that our present workforce is enough to maintain it. A french proverb says that "L'enfer est pavé de bonnes intentions" (Hell is paved with good intents) ; I think that we have such a case...

Now, for the cleanup of Python 2-specific idioms, I suspect that this will take a long time. And defining the set of Python 2 expressions that would be advantageously rewritten with (different) Pyhon 3 idioms is by no way an easy task. I would suggest to keep it as an informal goal, and to treat it incrementally.

HTH,

[William Stein]

On Tue, Jan 21, 2020 at 9:49 AM Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
>
> I second that. Unless someone presents us with a pressing case of a large Python 2-based library unportable to Python 3 (and therefore already in a dead-end), our limited workforce is better employed to maintain our Python 3 Sage.

I'm just going to endorse this decision yet again based on (for me)
new evidence:

1. I personally spent a lot of frustrating time last week working on
some Python code to make it worth well with both Python2 and Python3
versions of Sage. The code involved a lot of string manipulation
(display of output, sending data over networks connections, etc.), and
it was a *very* painful experience with expressions that are valid in
both Python2 and Python3, but behave differently in each, etc. I
would not wish such work on anybody.

2. Also I read the blog linked to from
https://news.ycombinator.com/item?id=22036773 which talks about how
Mercurial's decision to support Python2 so long made their support of
Python3 more difficult, and continues to have a significant negative
impact moving forward.

So I endorse Sage developers using their limited resources to move
Sage forward to be Python 3 only, and let old versions of Sage provide
backward compatibility for people with lots of Python2 Sage code.

-- William

--
William (http://wstein.org)

[E. Madison Bray]

On Tue, Jan 21, 2020 at 5:19 PM Timo Kaufmann <eisf...@gmail.com> wrote:
>
>
>
> Am Freitag, 17. Januar 2020 16:07:08 UTC+1 schrieb E. Madison Bray:
>>
>> On Wed, Jan 15, 2020 at 2:37 AM Matthias Koeppe
>> <matthia...@gmail.com> wrote:
>> >
>> > On Tuesday, January 14, 2020 at 12:01:42 PM UTC-5, Frédéric Chapoton wrote:
>> >>
>> >> So here is my proposal.
>> >>
>> >> * Starting from now, we allow ourselves to move on, using 9.1 betas and further releases for external python3 updates, including switch to ipython7, which seems to me the most urgent matter. But we also do not introduce python3-only code in our own code base if we can avoid it.
>>
>> I'm not exactly sure what you mean by this, but it might be agreeable
>> to me: I'm completely okay with adding Python 3 only dependencies and
>> even *code/features* so long as it's done without breaking
>> backwards-compatibility. When it comes to sage-the-distribution
>> (which I sense is where most of this friction is coming from, and yet
>> another reason to better separate sage from the
>> sage-the-distribution), if there are dependencies you want to update
>> that are Python 3-only I'd say go for it, but make it a separate SPKG
>> so that previous versions of the dependency can still work on Python 2
>> builds.
>
>
> As others have said already, supporting both python2 and python3 dependencies is much more work than only supporting python2 or only supporting python3 versions.

I don't think it is. I think that maintaining Python 2 support for
now (and gradually phasing it out) is the path of least resistance.

> I think dropping pyhon2 support with the next release remains the best option. Even better of course would be to keep the option for bugfix releases for 8.9 / 9.0. That would be a nice thing to have in general, but we could also just make an exception for this one release.

Yes, that would be better.

[E. Madison Bray]

On Fri, Jan 10, 2020 at 1:03 AM Volker Braun <vbrau...@gmail.com> wrote:
>
> * I think its not too difficult to write code that is Python 2.7 + 3.x (for high enough x) compatible, so its not a super pressing issue
> * We do have a Python 2 buildbot to test for regressions
> * For semver reasons we should drop Python 2.7 support in Sage 10, not 9.1
>
> Having said that, I'm fine with an accelerated Sage 9 -> Sage 10 schedule, maybe a month or two instead of the usual 3-4. Though first we should take the opportunity and see if there are any outstanding Python 3 bugs now that we have more data. For example it would be nice if a build with SAGE_DEBUG=yes would pass tests. There are a few more regressions, e.g. #28817

Since this thread had gone on quite long and it has perhaps gotten
lost, I would just like to once again endorse Volker's suggestion
here. We have already put enormous effort (hundreds of hours) into
making Sage Python 2/3 compatible and making it relatively easy to
write compatible code (and most new code will not even encounter these
compatibility issues).

Let us have a short release cycle for 9.1 and add a prominent
deprecation message in 9.1 for Python 2 builds, then Sage 10.0 can
implicitly start dropping support for Python 2. If there are some
patches that are needed (e.g. to support ipython7) they will have to
be applied eventually anyways so if some niche packing system must
apply it for Sage 9.1 I see little harm.

[Matthias Koeppe]

I have created #29141 (Meta-ticket: Upgrades and other changes that require dropping py2 support or separate package versions for py2/py3) to summarize this thread. 

[Matthias Koeppe]

Here's the link to the ticket for convenience. https://trac.sagemath.org/ticket/29141