print to python3

[Frédéric Chapoton]

Hello,

I am trying to move toward using the python3 syntax for print.

This is done by small chunks, using a lot of tiny tickets, according the folders in sage, mostly.

These tickets can be found at the bottom of

http://trac.sagemath.org/query?status=!closed&component=python3

If some of you could find time to review one of those, that would help to advance the change.

Many thanks to those that have already reviewed some of these.

cheers,

Frederic

[Dima Pasechnik]

I already commented on one of these tickets that I do not understand why `from __future__ import print_function`

is needed everywhere.

Indeed, most modules do not do nontrivial printing anywhere, except perhaps docstrings

(and the latter are not affected by that import anyway).

Dima

[Travis Scrimshaw]

   Is it strictly needed, no. However, it allows us to be compatible with Python3 and it helps prevent anyone in the future from adding code that is only works in Python2 wrt print statements. Furthermore, most of the docstrings can be made 2/3 compliant and don't need "fancy" print_function stuff. At some point (perhaps when we start 7.3.beta0?), we will change our interactive shell and doctesting framework to use the new print statements.

TL;DR, it doesn't hurt to add it.

Best,
Travis

[Frédéric Chapoton]

Hello,

I think we have made a good step towards using the python 3 print style. There are many tickets with positive review already done on the way. Let us hope that they will enter sage soon. Once this is done, the plan is to add "from __future__ import print_function" to src/sage/all.py.

http://trac.sagemath.org/query?status=!closed&component=python3

If you have time to review one of the tickets still open, please do so. Again, thanks a lot to all the people that have reviewed all these tickets.

I have a question: in sage/local/bin, at least two .py files seem obviously not py3 compatible, namely smtpd.py and rst2odt_prepstyles.py.

Should I try to fix these, or should we rather try to look if somebody has fixed them somewhere else ?

Frederic

[Jeroen Demeyer]

On 2016-05-16 11:34, Frédéric Chapoton wrote:
> Once this is done, the plan is
> to add "from __future__ import print_function" to src/sage/all.py.

Why only "once this is done". What makes that file so special?

[Frédéric Chapoton]

Because if you do that now, it triggers doctests failure all over the place.

[Frédéric Chapoton]

Hello,

concerning changing the behaviour of print to python3 style,

There are still a few tickets needing review:

http://trac.sagemath.org/query?status=!closed&component=python3

Frederic

Le mardi 3 mai 2016 20:50:21 UTC+2, Frédéric Chapoton a écrit :

[Frédéric Chapoton]

Hello everybody,

we are now almost ready to try to use the Python 3 print() function everywhere in Sage: doctests, command-line, notebook.

This big switch is the aim of ticket http://trac.sagemath.org/ticket/20668. It still has to wait for an update of sagenb and for the next beta.

This is an important change, that will likely break a lot of code by our users. So we have to discuss if we all agree to make this change right now, and if yes, how

to advertise for it and to warn everybody very loud, and to what documents to point to help people adapt their code.

I propose to add a message below the banner saying something like "WARNING: the behaviour of print has been changed. It is now a function: use print("text")"

with a link to a webpage (on the wiki ?) explaining the reasons for the change and how to cope with it

Please express yourself on the subject.

Frederic

[John Cremona]

On 24 May 2016 at 12:33, Frédéric Chapoton <fchap...@gmail.com> wrote:
> Hello everybody,
>
> we are now almost ready to try to use the Python 3 print() function
> everywhere in Sage: doctests, command-line, notebook.
> This big switch is the aim of ticket http://trac.sagemath.org/ticket/20668.
> It still has to wait for an update of sagenb and for the next beta.
>
> This is an important change, that will likely break a lot of code by our
> users. So we have to discuss if we all agree to make this change right now,
> and if yes, how
> to advertise for it and to warn everybody very loud, and to what documents
> to point to help people adapt their code.
>
> I propose to add a message below the banner saying something like "WARNING:
> the behaviour of print has been changed. It is now a function: use
> print("text")"
> with a link to a webpage (on the wiki ?) explaining the reasons for the
> change and how to cope with it
>

Would it be possible for

sage: a=3
sage: print a

to work on the command line, using the preparser? I think that would
make users a lot less annoyed. I *only* mean on the command line and
notebook cells, not in code (*.py) files.

John

> Please express yourself on the subject.
>
> Frederic
>
>
> Le samedi 21 mai 2016 09:09:26 UTC+2, Frédéric Chapoton a écrit :
>>
>> Hello,
>>
>> concerning changing the behaviour of print to python3 style,
>>
>> There are still a few tickets needing review:
>>
>> http://trac.sagemath.org/query?status=!closed&component=python3
>>
>> Frederic
>>
>>
>> Le mardi 3 mai 2016 20:50:21 UTC+2, Frédéric Chapoton a écrit :
>>>
>>> Hello,
>>>
>>> I am trying to move toward using the python3 syntax for print.
>>>
>>> This is done by small chunks, using a lot of tiny tickets, according the
>>> folders in sage, mostly.
>>> These tickets can be found at the bottom of
>>>
>>> http://trac.sagemath.org/query?status=!closed&component=python3
>>>
>>> If some of you could find time to review one of those, that would help to
>>> advance the change.
>>>
>>> Many thanks to those that have already reviewed some of these.
>>>
>>> cheers,
>>> Frederic
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[Jeroen Demeyer]

On 2016-05-24 14:06, John Cremona wrote:
> Would it be possible for
>
> sage: a=3
> sage: print a
>
> to work on the command line, using the preparser?

That already works today.

If you want that to continue working, it's better to not switch to
Python 3 style printing. It's kind of silly to change Sage to use Python
3 printing and at the same time add a hack to support Python 2 printing.
Better stick with Python 2 printing then...

[Jeroen Demeyer]

On 2016-05-24 14:06, John Cremona wrote:

> in code (*.py) files.

For .py code files, there is not really an issue since those can just do
from __future__ import ...

[John Cremona]

On 24 May 2016 at 13:17, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
> On 2016-05-24 14:06, John Cremona wrote:
>>
>> Would it be possible for
>>
>> sage: a=3
>> sage: print a
>>
>> to work on the command line, using the preparser?
>
>
> That already works today.

But would it still work after a total conversion of Sage to Python 3?

Most Sage users do not care a fig about Python versions, but will care
if after 10 years of being able to type "print a" they are suddenly
forced into typing "(print(a)". Iwas thinking about this from a user
perspective not a developer perspective.

John

>
> If you want that to continue working, it's better to not switch to Python 3
> style printing. It's kind of silly to change Sage to use Python 3 printing
> and at the same time add a hack to support Python 2 printing. Better stick
> with Python 2 printing then...
>
>

[Jori Mäntysalo]

On Tue, 24 May 2016, John Cremona wrote:

> Most Sage users do not care a fig about Python versions, but will care
> if after 10 years of being able to type "print a" they are suddenly
> forced into typing "(print(a)". Iwas thinking about this from a user
> perspective not a developer perspective.

You have very good perspective.

Can we add a deprecation warning to old-style print?

--
Jori Mäntysalo

[Vincent Delecroix]

On 24/05/16 07:48, John Cremona wrote:
> On 24 May 2016 at 13:17, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>> On 2016-05-24 14:06, John Cremona wrote:
>>>
>>> Would it be possible for
>>>
>>> sage: a=3
>>> sage: print a
>>>
>>> to work on the command line, using the preparser?
>>
>>
>> That already works today.
>
> But would it still work after a total conversion of Sage to Python 3?
>
> Most Sage users do not care a fig about Python versions, but will care
> if after 10 years of being able to type "print a" they are suddenly
> forced into typing "(print(a)". Iwas thinking about this from a user
> perspective not a developer perspective.
>

I agree with John. We should add a DeprecationWarning in the Sage
preparser right now for "print X, Y, Z" and make it such that "print(X,
Y, Z)" really behaves like Python 3.

Nothing will change with .py files. But in the console, notebook and
.sage files users would be noticed.

Vincent

[Travis Scrimshaw]

On Tuesday, May 24, 2016 at 7:55:49 AM UTC-5, vdelecroix wrote:

On 24/05/16 07:48, John Cremona wrote:
> On 24 May 2016 at 13:17, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>> On 2016-05-24 14:06, John Cremona wrote:
>>>
>>> Would it be possible for
>>>
>>> sage: a=3
>>> sage: print a
>>>
>>> to work on the command line, using the preparser?
>>
>>
>> That already works today.
>
> But would it still work after a total conversion of Sage to Python 3?
>
> Most Sage users do not care a fig about Python versions, but will care
> if after 10 years of being able to type "print a" they are suddenly
> forced into typing "(print(a)".  Iwas thinking about this from a user
> perspective not a developer perspective.
>

I agree with John. We should add a DeprecationWarning in the Sage
preparser right now for "print X, Y, Z" and make it such that "print(X,
Y, Z)" really behaves like Python 3.

Nothing will change with .py files. But in the console, notebook and
.sage files users would be noticed.

I also agree with John's suggestion as a temporary measure which warns users (every time, not just the first) that it will no longer be supported, which will include anyones .sage files too.

Also, I think the longer we delay on becoming Python3 compliant, the more we are losing in potential users and developers who are starting by learning Python3.

Best,
Travis

 

[Jeroen Demeyer]

On 2016-05-24 15:27, Travis Scrimshaw wrote:
> every time, not just the first

-1. We should not annoy our users with endless deprecation warnings.

[John Cremona]

We seem to be stuck. The possibilities are

(1) make "print a" just stop working at some point (maybe not yet)
(2) as (1) but with a deprecation warning
(3) continue to support "print a" for ever.

All three have had some negative votes!

Jeroen, I also do not like "endless" deprecation warnings, but if it
was limited to one per session (assuming that is technically possible)
it would not be so bad?

John

[William Stein]

Breaking "print a" will cause a truly epic level of pain to our users for no real gain...  So much so that probably no matter what is decided here I would fork sage to add handling this to the pre-processor for sage on SMC.  I'm here from endless users (eg each year at the sage booth) about minor deprecations that piss them off.  This print thing will be horrible.    Truly horrible. 

I programmed in python3 many of my waking hours for the last 2 weeks and even last night I was still  making mistakes with print.   I'm absolutely certain users will have similar trouble, both adapting their own code and in changing.   It's just psychology.  

(Personally i really like Python3 and the new print function. However, I (and you) are not the only user of sage.)

William

--
Sent from my massive iPhone 6 plus.

[Nils Bruin]

On Tuesday, May 24, 2016 at 10:21:34 AM UTC-7, William wrote:

Breaking "print a" will cause a truly epic level of pain to our users for no real gain...  So much so that probably no matter what is decided here I would fork sage to add handling this to the pre-processor for sage on SMC.

Preprocessing "print" in a robust and reliable way will be a bit of a challenge, though. In python 3, this is now valid:

(print(1), print(2))

and this too:

>>> print=2
>>> print-1
1

I guess in the sage command line, "print" would still be advertised as a reserved word, so we wouldn't have to handle the latter scenario, but figuring out where the parentheses go for other uses of print is going to be tricky. I'm not sure something that is just regex based will do the job.

[William Stein]

Damn reality. I had imagined pre-parsing only

print [whitespace] [non whitespace that doesn't start with a paren]

and nothing else. If the user uses print as a variable, this would
break their code. I'm OK with that.



William

[Johan S. R. Nielsen]

> Breaking "print a" will cause a truly epic level of pain to our users for
> no real gain... So much so that probably no matter what is decided here I
> would fork sage to add handling this to the pre-processor for sage on SMC.
> I'm here from endless users (eg each year at the sage booth) about minor
> deprecations that piss them off. This print thing will be horrible.
> Truly horrible.

> ...

> (Personally i really like Python3 and the new print function. However, I
> (and you) are not the only user of sage.)

I also think that this will annoy a lot of users. On the other hand,
Python 3 is already old, and the transition has to come at some point.
In 2020, at the latest we shouldn't be supporting Python 2 at all.

Perhaps a tolerable amount of pain would be to do the deprecation in
several steps:

1) For year, issue only a deprecation *the first time* "print a" is used
in a session.
2) After that, for X years, issue a deprecation warning *every time*
"print a" is used.
3) After that, remove support for "print a" completely.

Best,
Johan

>> <javascript:;>> wrote:
>> > On 2016-05-24 15:27, Travis Scrimshaw wrote:
>> >>
>> >> every time, not just the first
>> >
>> >
>> > -1. We should not annoy our users with endless deprecation warnings.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "sage-devel" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an

>> > email to sage-devel+...@googlegroups.com <javascript:;>.

>> > To post to this group, send email to sage-...@googlegroups.com

>> <javascript:;>.

>> > Visit this group at https://groups.google.com/group/sage-devel.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sage-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to sage-devel+...@googlegroups.com <javascript:;>.

>> To post to this group, send email to sage-...@googlegroups.com

>> <javascript:;>.

>> Visit this group at https://groups.google.com/group/sage-devel.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> Sent from my massive iPhone 6 plus.

--

[William Stein]

On Tue, May 24, 2016 at 11:27 AM, Johan S. R. Nielsen
<santa...@gmail.com> wrote:
>> Breaking "print a" will cause a truly epic level of pain to our users for
>> no real gain... So much so that probably no matter what is decided here I
>> would fork sage to add handling this to the pre-processor for sage on SMC.
>> I'm here from endless users (eg each year at the sage booth) about minor
>> deprecations that piss them off. This print thing will be horrible.
>> Truly horrible.
>> ...
>> (Personally i really like Python3 and the new print function. However, I
>> (and you) are not the only user of sage.)
>
> I also think that this will annoy a lot of users. On the other hand,
> Python 3 is already old, and the transition has to come at some point.
> In 2020, at the latest we shouldn't be supporting Python 2 at all.
>
> Perhaps a tolerable amount of pain would be to do the deprecation in
> several steps:
>
> 1) For year, issue only a deprecation *the first time* "print a" is used
> in a session.
> 2) After that, for X years, issue a deprecation warning *every time*
> "print a" is used.
> 3) After that, remove support for "print a" completely.
>
> Best,
> Johan

I support your proposal above, with X=1. I don't want to have the
print statement forever.

-- William

> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.

> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--

William (http://wstein.org)

[Jori Mäntysalo]

On Tue, 24 May 2016, Johan S. R. Nielsen wrote:

> 1) For year, issue only a deprecation *the first time* "print a" is used
> in a session.
> 2) After that, for X years, issue a deprecation warning *every time*
> "print a" is used.
> 3) After that, remove support for "print a" completely.

0) For a year(?), distribute version of Sage where every example uses only
print(). Contact writers of Sage textbook etc.

--
Jori Mäntysalo

[Frédéric Chapoton]

I agree with Johann's proposal, with X=1 year  as William proposed.

The doc will very soon already have print() everywhere.

I have contacted at least one author of the French book. They want to make a revised edition and an English traduction, and they will use print().

Who is volunteering to write the parser ?

Frederic

[Jori Mäntysalo]

On Tue, 24 May 2016, Frédéric Chapoton wrote:

> I agree with Johann's proposal, with X=1 year  as William proposed.
> The doc will very soon already have print() everywhere.

In beta, yes. But 7.2 was just released, so it will take time for 7.3 to
be out.

> I have contacted at least one author of the French book. They want to make a
> revised edition and an English traduction, and they will use print().

Very good!

Still one idea to consider: Maybe the parser could be optional, enabled by
default. If someone will start a new course to fresh students with new
server, maybe disabling print-parser makes sense.

--
Jori Mäntysalo

[William Stein]

Hi,

I've been thinking more, and I'm really disturbed by this piecemeal
approach to getting Python 3, as least as far as it impacts *users*
(for developers it is great).

As mentioned before, I've been writing tons of code using Python 3 for
the last two weeks. There are *many* subtle ways in which this
python2 --> python3 switch will flat out break people's code/brains,
which go far beyond just the print statement. Frankly, I think
trying to gently migrate *end users/college teachers* in multiple
steps is just going to make matter worse.

For example, I'm finally using print as a function. Cool. Now (this
just happened a minute ago), I write:

[blah... for a,b in x.iteritems()]

where x is a dictionary. This is/was the optimal way to iterate over
the key/values of a dictionary in Python2. In Python3, iteritems is
just completely gone. There's surely tons of other things like this.

We need a real strategy for migrating users to Python3, and definitely
not some half-way thing that deals only with the print statement.

** If we break compatibility in the *slightest* for Python3, I think
it should only happen *once*. **

The last thing we want is:

- I upgraded to Sage-7.3 and *all* my 100s of worksheets I use in
teaching broke due to print statements. I spent 10 hours going
through and fixing them all -- ugh. Misery.

Then...

- I upgraded to sage-7.5 and all my code broke again due to (some
other python3 feature we enable)... I spent 10 hours going through
and fixing them all -- ugh. Misery and pain.

Then...

- I upgraded to Sage-8.0 and all my code broke badly again due to
Sage switching to Python3! I spent hours rewriting my code a third
time, running scripts (like the 2-to-3 convertor, but preparser
aware?), etc.

If we are going to force users to rewrite everything to work, then
let's do it all at once, rather than multiple times. And automate it
as much as possible.

Because of SageMathCloud, I know for a fact that lots of teachers
write lots of code using Jupyter notebooks and Sage worksheets, which
of course has print statements all over and other python2-isms. They
don't have the time to deal with stuff like this. If anything
involving maintaining their work takes "10 hours", they will like just
not do it ever. If they use SMC, I'll probably just make it so they
can stay with sage-7.2 "forever", just like people stick with gap3 and
python2 forever. However, if we make switching from Python2 to
Python3 trivial/smooth/easy for end users, they'll all switch no
problem and love it.

-- William

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.

> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--

William (http://wstein.org)

[Nils Bruin]

On Tuesday, May 24, 2016 at 1:56:56 PM UTC-7, William wrote:

Hi,

I've been thinking more, and I'm really disturbed by this piecemeal
approach to getting Python 3, as least as far as it impacts *users*
(for developers it is great).

[...]

For example, I'm finally using print as a function. Cool.  Now (this
just happened a minute ago), I write:

  [blah... for a,b in x.iteritems()]

 
I think we're just bumping into the same issue that makes the python 2->3 transition disruptive for everyone. If we could come up with a solution to smoothly transition from one to the other, we'd have a way to smoothly transition code from python 2 to python 3 (or in other words: have enough of a compatibility layer on top of python (2+i) to run python (3-i) [for i=0 or 1] ). Hasn't this been the holy grail the python community has been looking for?

I think this suggests we *won't* be able to find a way to do this satisfactorily. So I expect we're just stuck with a relatively sudden, not-so-compatible transition from sage-7.* to sage-8.0 where we have python 3 underneath instead of python 2.

For now we could try:
 - to get the codebase into a shape where the transition is doable (but leave user interface alone)
 - work on tools that help people to migrate from sage/py2 to sage/py3

For people who use notebook servers, one could easily run a jupyter notebook server that supports both a sage-7.* and a sage-8.0 kernel, providing a platform to migrate from one to the other with relative ease (this will probably be a little harder with sagenb, but not by much if the pathname to the other sage kernel is provided)

Depending on resources it might be good to continue "supporting" a sage-7.* for a while when the python 3 sage-8.0 has come out. I'd expect new features would very quickly cease to be backported to sage-7.*, hopefully providing a positive incentive for people to migrate to sage-8.0.

If William can get people for this, it might be the kind of thing where money actually would help (not just to produce sage-8.0/py3, but also to reduce the trauma to the community from the transition)

[Fernando Perez]

On Tue, May 24, 2016 at 1:56 PM, William Stein <wst...@gmail.com> wrote:

We need a real strategy for migrating users to Python3, and definitely
not some half-way thing that deals only with the print statement.

Just lurking here, but that was precisely the thought that came to my mind.

For reference, what we did in IPython (and I really mean the IPython kernel, not the notebook/Jupyter/etc) and that has ultimately worked very well (not that it wasn't a lot of effort in the first place):

- Update the IPython kernel to run from a single source in python 2 or 3.

- Provide users an easy way to install IPython as a python2 or python3 kernel for Jupyter.

This decouples *our* development from the *user's* code.  They can choose, when they each see fit, whether to run a python2 or a python3 stack. Since all the major scientific python libraries are py2/3 compatible, the choice is up to them.

This means that, when they choose to update to py3, they pay the price as they need to, and they can pay that cost even piecemeal, moving one project over but not another.

I suspect that flat out mandating a hard sage-py2 to sage-py3 transition at any given point in time will not work at all, it will cost sage users and will effectively freeze many of them in the sage-py2 version forever.

Maintaining a py2/3 single-source codebase takes some extra work (and diligent testing on both platforms) but it's not impossible.  I really think this is the only viable path for sage if you want to retain your user base and help them make the transition across the py2/3 divide.

Cheers

f

--

Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

[William Stein]

Fernando,

Thanks for your post! The assumption in this whole thread is that
Sage supports either python2 or python3, and not both. Thanks for
questioning this assumption and pointing out that "all the major
scientific python libraries are py2/3 compatible". This reminds me
again of this post [1] which is about sympy moving to be "python3
only"...

[1] https://news.ycombinator.com/item?id=11738470

[Jori Mäntysalo]

On Tue, 24 May 2016, Nils Bruin wrote:

> For people who use notebook servers, one could easily run a jupyter notebook
> server that supports both a sage-7.* and a sage-8.0 kernel, providing a
> platform to migrate from one to the other with relative ease (this will
> probably be a little harder with sagenb, but not by much if the pathname to
> the other sage kernel is provided)

Already I can use SageNB as an interface for, say, plain Python or R or
GAP. So is it possible to add "Sage (compat)" to that list? Or should it
be a checkbox with a link explaining what are main differences?

--
Jori Mäntysalo

[Jeroen Demeyer]

On 2016-05-24 20:20, Nils Bruin wrote:
> figuring out where the parentheses go for other uses of print is going
> to be tricky. I'm not sure something that is just regex based will do
> the job.

It doesn't have to be perfect. It's already easily possible to fool the
preparser with strange constructions. But in practice, that's not a big
deal.

[Jeroen Demeyer]

On 2016-05-24 20:27, Johan S. R. Nielsen wrote:
> 2) After that, for X years, issue a deprecation warning *every time*
> "print a" is used.

As I said before, I don't like this part.

[Dima Pasechnik]

In fact, I would not be surprised (I'm in fact willing to bet that this will happen) if python2 were around for another 20-30 years, perhaps forked off the main python development.

Suppressing a language (or a dialect) with a sizeable following is always a pain, and politically explosive, be it a programming or a natural language (for the latter: e.g. despite centuries of efforts of politicians ( :-( ), Celtic languages are still alive, to an extent; Swiss German is alive and well and not going away any time soon, etc...).

Dima

PS. as someone who functions in largely foreign language environments for most of my adult life, I am perhaps more aware of this than most people. :-)

[Simon King]

Hi William,

On 2016-05-24, William Stein <wst...@gmail.com> wrote:
> The last thing we want is:
>
> - I upgraded to Sage-7.3 and *all* my 100s of worksheets I use in
> teaching broke due to print statements. I spent 10 hours going
> through and fixing them all -- ugh. Misery.
>
> Then...
>
> - I upgraded to sage-7.5 and all my code broke again due to (some
> other python3 feature we enable)... I spent 10 hours going through
> and fixing them all -- ugh. Misery and pain.
>
> Then...
>
> - I upgraded to Sage-8.0 and all my code broke badly again due to
> Sage switching to Python3! I spent hours rewriting my code a third
> time, running scripts (like the 2-to-3 convertor, but preparser
> aware?), etc.

+1

I found it quite annoying when my spkg stopped working for some
silly reason, I fixed it, then it broke for another reason, etc. That
frustrating experience shouldn't happen too often.

Cheers,
Simon

[John Cremona]

This looks helpful: http://python-future.org/compatible_idioms.html

I was looking for a simple-minded person's guide to what they would
actually have to change to make the transition. There is a lot of
ranting out there and stuff wriiten for people for whom programming
seems more and end in itself than a means to an end.

Apart from print there is string stuff (not very relevant for users or
most ordinary developers), iteration through dictionaries, xrange &
range, map & imap, zip & izip, and that is about it (filtering by
things which I think *I* would need to know and change for Sage
development.

John

[Jeroen Demeyer]

On 2016-05-25 01:52, Nils Bruin wrote:
> Hasn't this been the holy grail the
> python community has been looking for?

I don't know what you mean with "the Python community" but it's
certainly not a priority for upstream CPython. If CPython really wanted
to make it easier to port Python 2 code to Python 3, then Python 3 dicts
would still have an iteritems() method for example.

[Frédéric Chapoton]

Thanks for your reactions. 

Let me try to summarize. We seem to more or less agree that we should not repeatedly break the code of all users of sage. To avoid that, it would be better to switch completely to python3 in just one major change. And then there will be something like the "last py2 release" and the "first py3 release".

Our current aim would be to reach a code that is both compatible with py2 and py3. There is certainly room for progress in that direction. If somebody could provide a precise

technical guide on how to try to build sage using py3, I would appreciate. My own attemps are not going very far.

I will therefore now forget about ticket 20668 . I nevertheless propose that we require that every single new .py(x) file added to sage from now on contains the line

"from _future__ import division, print_function, absolute_import" in order to prevent backward moves.

Frederic

Le mardi 24 mai 2016 13:33:37 UTC+2, Frédéric Chapoton a écrit :

Hello everybody,

we are now almost ready to try to use the Python 3 print() function everywhere in Sage: doctests, command-line, notebook.

This big switch is the aim of ticket http://trac.sagemath.org/ticket/20668. It still has to wait for an update of sagenb and for the next beta.

This is an important change, that will likely break a lot of code by our users. So we have to discuss if we all agree to make this change right now, and if yes, how

to advertise for it and to warn everybody very loud, and to what documents to point to help people adapt their code.

I propose to add a message below the banner saying something like "WARNING: the behaviour of print has been changed. It is now a function: use print("text")"

with a link to a webpage (on the wiki ?) explaining the reasons for the change and how to cope with it

Please express yourself on the subject.

Frederic

Le samedi 21 mai 2016 09:09:26 UTC+2, Frédéric Chapoton a écrit :

Hello,

concerning changing the behaviour of print to python3 style,

There are still a few tickets needing review:

http://trac.sagemath.org/query?status=!closed&component=python3

Frederic

[Johan S. R. Nielsen]

> Let me try to summarize. We seem to more or less agree that we should not
> repeatedly break the code of all users of sage. To avoid that, it would be
> better to switch completely to python3 in just one major change. And then
> there will be something like the "last py2 release" and the "first py3
> release".

Does this entail that in this "last py2 release", we insert deprecation
warnings for all py2 \setminus py3 features that we possibly can,
simultaneously? That way, users would only change all their code that
one time, to get rid of all the deprecation warnings. And then their
code would (usually) work in the py3 version.

I'm intrigued by what Fernando wrote, namely supporting both py2 and py3
for a long time/forever. But I don't know enough about whether the
development overhead would be big -- to me, it sounds like it.

Best,
Johan

[kcrisman]

Most Sage users do not care a fig about Python versions, but will care
if after 10 years of being able to type "print a" they are suddenly
forced into typing "(print(a)".  Iwas thinking about this from a user
perspective not a developer perspective.

Bingo. 

> Also, I think the longer we delay on becoming Python3 compliant, the more we are losing in potential users and developers who are starting by learning Python3.

That's ridiculous.  The changes between P2 and P3, while nontrivial, are much more casual for most users.  Where we will lose people is by once again following the common (not universal) open-source mantra of "if it's good for developers, it must be good for users".

As William says,

> I'm here from endless users (eg each year at the sage booth) about minor deprecations that piss them off.  This print thing will be horrible. 

Yup.

> However, if we make switching from Python2 to Python3 trivial/smooth/easy for end users, they'll all switch no problem and love it. 

Double yup!  Though perhaps harder.  On deprecation (if that is possible), I would personally vote for X = 2 as many people don't teach the same course every year, but I know I might be in the minority on that.

I also like Fernando's post - my assumption in our having py2/py3 package availability was that we were planning on supporting both, but maybe that is not possible/too crazy.

[Harald Schilly]

On Tuesday, May 24, 2016 at 5:48:59 PM UTC+2, John Cremona wrote:

We seem to be stuck.

I haven't read all the posting here, but just to throw in an idea that might not have been mentioned before: we could use this 2to3 utility in the preparsing step. I.e. an extended pipeline like input → preparser → 2to3 → evaluation.

Below is a sample code that does that, which is my just showing me that this is doable from within SageMath (running on SMC, SageMath 6.10)

%python
from lib2to3 import refactor
FIXERS_2TO3 = refactor.get_fixers_from_package("lib2to3.fixes")
rt = refactor.RefactoringTool(FIXERS_2TO3)
processed = preparse("""\
print "hello"
for k, v in d.iteritems():
    print k, v
f(x) = 1+x^2
[blah(a,b) for a,b in x.iteritems()]
print range(10)
print dict.keys()
""")
tree = rt.refactor_string(processed, "SageMath2to3")
print(unicode(tree))

which produces

print("hello")
for k, v in d.items():
    print(k, v)
__tmp__=var("x"); f = symbolic_expression(Integer(1)+x**Integer(2)).function(x)
[blah(a,b) for a,b in x.items()]
print(list(range(Integer(10))))
print(list(dict.keys()))

Notable:

* didn't destroy or got confused by SageMath's preparse step (tested with this f(x) = ... and Integer)

* iteritems → items

* did wrap list( ... ) around these new generators, which are different from py2 (yes, that's not always intelligent, but hey that's the tradeoff)

* did fix williams example

Missing:

* As mentioned somewhere here, the print function needs fine-tuning to be more equivalent, but that's doable with a bit of heuristics.

My perspective on that is, that we could use this as a way to keep the UI in a long-term compatibility mode. At first, it's a default, but then we can figure out a way to make this somehow prominent and urge everyone to think about switching this compatibility mode layer off. Of course, my small test here is also by no means a full test. There are likely some hurdles left.

-- harald

[Nils Bruin]

On Wednesday, May 25, 2016 at 8:23:07 AM UTC-7, Harald Schilly wrote:

I haven't read all the posting here, but just to throw in an idea that might not have been mentioned before: we could use this 2to3 utility in the preparsing step. I.e. an extended pipeline like input → preparser → 2to3 → evaluation.

That is a very attractive idea, because we get to reuse a lot of careful work other people have already done on the subject.

It's also quite conceivable that the 2to3 step is not *that* much more expensive than our current preparser. In that case it might be worth considering if we can integrate our preparser steps into the lib2to3.refactor framework: it is a full parser, so it will be a lot more powerful and robust than the regex/substitution approach taken now. We'd also get to formalize the sage grammar extensions into a formal grammar description.

[Harald Schilly]

On Wed, May 25, 2016 at 6:28 PM, Nils Bruin <nbr...@sfu.ca> wrote:
> In that case it might be worth considering if we can integrate our preparser
> steps into the lib2to3.refactor framework:

I'm by far not an expert on parsers, but I think forking that 2to3
part in the standard library could work (i.e. a lot of careful work,
but then it's much more robust). Internally, I think this tool works
by being able to parse both python versions into a syntax tree --
that's done in a .c file, such that this grammar definition is being
compiled to run quickly. Then, there are many python files in fixes/
which are doing the transformations of the parsed tree. There is also
a fix_print.py, which would need that little bit of patching to handle
the case of tuples vs. concatenated strings better. SageMath specific
transformations would be additional fix_*.py transformations.

-- h

[Erik Bray]

I like the sound of this too. lib2to3 can be pretty powerful. Its
biggest downside is lack of documentation (at least, last I tried to
do anything with it). For example it can be hard to figure out how to
write a fixer.

Interest in lib2to3 has also declined as it became clear, through
practice, that it was not a long-term viable solution for supporting
Python 2 and 3 simultaneously, and that for any reasonably complicated
project it didn't do enough to be worth the trouble.

But for this use of translating statements typed into a REPL I think
it could be very useful. Especially if we only enable a limited
subset of fixers that are easy to reason about (e.g. print, iteritems,
etc.)

[Fernando Perez]

On Tue, May 24, 2016 at 9:09 PM, William Stein <wst...@gmail.com> wrote:

Thanks for your post!  The assumption in this whole thread is that
Sage supports either python2 or python3, and not both.  Thanks for
questioning this assumption and pointing out that "all the major
scientific python libraries are py2/3 compatible".    This reminds me
again of this post [1] which is about sympy moving to be "python3
only"...

[1] https://news.ycombinator.com/item?id=11738470

Yup, just to be clear, those libraries are *currently* py2/3 compatible, but indeed we have the plan you pointed to of gradually dropping py2.  I should have been more explicit, the idea is:

1. Period of single-source py2/3 compatibility.  This lets users move at their own pace.  In the case of Sage, this could be restricted to just the parts of Sage that execute user code, you could move to py3 as a requirement for installation/development, for example, if that made sense.  You just don't impose that on user code quite yet.  Make it really easy to install either "kernel" for users, and encode that metadata in sage worksheets.

2. At some point, make a well-publicized, last release of this version that maintains the py2/3 option.

3. From then on, users can either start using the new py3 one or continue using py2 but they do lose new features.

4. Maintain, at least for a while, backporting of critical/security fixes to the py2 version, making only point releases.

So this is not a "maintain py2 forever" plan, but rather "give users options for a gradual, staggered transition at their own pace, while giving the Sage dev team a path forward to start using py3 more and more".

Cheers,

[Samuel Lelievre]

I am in favour of any of the suggested approaches which would

let users transition at their own pace.

[Kwankyu Lee]

I am wondering why no one mentions the "six" package to maintain a temporary py2/py3-compatible codebase. I have an experience of moving my own small project from python 2 to python 3 with the "six" package quite successfully. The strategy was to add "import six" at every file and make the file work on both python2 and python3. After a period of keeping the stable py2/py3-compatible version for a while, then I dropped supporting python2 and started replacing all the compatible code based on "six" to pure python3 code. Of course, compared with Sage, my own project was a tiny thing.