offer to help with Python 3 support

[Tim Graham]

I'm a developer on the Django team and we use Trac as our bug tracker. I'm interested in moving our infrastructure to Python 3, but need Trac to support Python 3 in order to do so. Django supports Python 2 and 3, so I have some experience in maintaining a code base that supports both. I started working on a patch (attached), but as it will be a non-trivial effort, I wanted to get an initial review and ensure this approach is agreeable to the Trac team. I also wanted to ensure that when I finish the patch, someone will be interested in reviewing and committing it relatively quickly so that it doesn't go stale. My strategy is to attempt to run the test suite on Python 3 and fix errors, while ensuring the tests pass on Python 2.7 as a I go. Thanks! Tim

[Peter Suter]

On 07.07.2015 20:51, Tim Graham wrote:

I'm a developer on the Django team and we use Trac as our bug tracker. I'm interested in moving our infrastructure to Python 3, but need Trac to support Python 3 in order to do so. Django supports Python 2 and 3, so I have some experience in maintaining a code base that supports both. I started working on a patch (attached), but as it will be a non-trivial effort, I wanted to get an initial review and ensure this approach is agreeable to the Trac team. I also wanted to ensure that when I finish the patch, someone will be interested in reviewing and committing it relatively quickly so that it doesn't go stale. My strategy is to attempt to run the test suite on Python 3 and fix errors, while ensuring the tests pass on Python 2.7 as a I go. Thanks! Tim

Sounds great. Thanks for helping out.
You are certainly right that this will be non-trivial. I'm sure there's some interest in such a patch, but it will take a lot of work and maybe the right timing.
If I remember correctly Trac 1.2  is still supposed to support Python 2.6. Only after that release would Trac 1.3 drop Py2.6 support. To avoid your work going stale you might want to adjust / time your strategy accordingly. Early and continued community involvement & reviews will surely help your chances of success.

You should definitely check this ticket if you haven't already:
http://trac.edgewall.org/ticket/10083

Working with the test suite for both Py2 and 3 is a very good idea I think. (But may not be sufficient.)

I see you used the six module. Would that be a new external dependency? (I think Trac has historically used as few dependencies as possible. If that's still a concern, it looks like six.py could be relatively easily "vendored".) Is there any point to using automatic conversion tools like http://python-future.org/automatic_conversion.html?

Some other details I noticed:
0666 (octal) and 0x666 (hex) etc. are not equivalent. Did you mean 0o666 etc.?
Your patch contains long lines, which are not allowed in Trac's code. For more information see:
http://trac.edgewall.org/wiki/TracDev/SubmittingPatches
http://trac.edgewall.org/wiki/TracDev/CodingStyle

Peter

[Ryan Ollos]

On Tue, Jul 7, 2015 at 2:51 PM, Tim Graham <timog...@gmail.com> wrote:

I'm a developer on the Django team and we use Trac as our bug tracker. I'm interested in moving our infrastructure to Python 3, but need Trac to support Python 3 in order to do so. Django supports Python 2 and 3, so I have some experience in maintaining a code base that supports both. I started working on a patch (attached), but as it will be a non-trivial effort, I wanted to get an initial review and ensure this approach is agreeable to the Trac team. I also wanted to ensure that when I finish the patch, someone will be interested in reviewing and committing it relatively quickly so that it doesn't go stale. My strategy is to attempt to run the test suite on Python 3 and fix errors, while ensuring the tests pass on Python 2.7 as a I go. Thanks! Tim

(Peter beat me to it, so there's some redundancy in my message :)

I appreciate your offer to help us move to Python 3. We are going to release Trac 1.2 soon. In the 1.3.x development line leading up to 1.4 we'll drop support for Python 2.6, only supporting Python 2.7 (1). Supporting Python 3.3 or 3.4 in 1.3.x is a possibility, but we haven't discussed it lately. The last time I recall discussing it was in (2) (see comment:30 in that ticket).

I expect it will be a major effort to have Trac support Python 3. Jun has done some work on it, so he can best comment on the effort required. You may want to read through #10083 (3) to see a list of concerns previously raised, and review the "python3" tickets (4). I recall there being concerns about some of our dependencies not supporting Python 3, such as MySQLdb.

With regard to the mechanics of proposing changes, you'll want to base your changes off the current trunk. If you create one monolithic path, it will be difficult for us to review, and will easily become stale, like you suggest. If you were to fork Trac in your favorite DVCS and start stacking up concise changes that lead to supporting Python 3, we can review and incorporate those incrementally. For example, you could submit a patch to fix #12046 (5). You could rebase regularly to avoid the changes that don't get pushed right away from becoming stale. 

I admit I'm not overly enthusiastic about supporting Python 2.7 and 3.x in the same codebase. I have wanted to make progress towards Python 3 so that we could jump to supporting a minimum of Python 3.3 or 3.4 in development leading up to Trac 1.6. Whether explicitly planned or not, we've been incrementing the minimum Python version supported with each major release (6). Since I've mentioned versions - more about our versioning scheme and roadmap can be found in (7).

(1) http://trac.edgewall.org/wiki/TracDev/ApiChanges/1.3

(2) http://trac.edgewall.org/ticket/11600#comment:20

(3) http://trac.edgewall.org/ticket/10083

(4) http://trac.edgewall.org/query?status=!closed&keywords=~python3 

(5) http://trac.edgewall.org/ticket/12046

(6) https://groups.google.com/d/msg/trac-dev/nkMUY_8ILF0/xdGVywGqTkYJ

(7) http://trac.edgewall.org/wiki/RoadMap#TracsRoadAhead

[Peter Suter]

On 08.07.2015 00:06, Ryan Ollos wrote:

I expect it will be a major effort to have Trac support Python 3. Jun has done some work on it, so he can best comment on the effort required. [...]

(2) http://trac.edgewall.org/ticket/11600#comment:20

By the way, Jun's work was linked in there:
https://github.com/jun66j5/trac/commits/python3

[Tim Graham]

Thanks for the feedback. As suggested, I'll try to work on some smaller patches at first to get familiar with contributing and wait until 1.2 is released before submitting any major changes.

Looking at Jun's work, it seems he is more or less reimplementing a subset of the functionality that six provides in `trac.util.compat`. I think adding six as a dependency or vendoring it (Django's solution) would save some work. Either approach is fine with me, just let me know what you think.

As far as dependencies go, I'll take a look as I go, but I'll just mention that Django now recommends the mysqlclient fork of MySQLdb as it's maintained and supports Python 3.

By the way, Django's philosophy regarding supporting both Python 2 and 3 is here: https://docs.djangoproject.com/en/dev/topics/python3/
Please give a read a let me know if you disagree with any of it so I don't head down the wrong path. I suspect there's no need to support Python 3.2, so we can probably omit the unicode_literals step, at least initially.

[RjOllos]

On Tuesday, July 7, 2015 at 5:36:19 PM UTC-7, Tim Graham wrote:

By the way, Django's philosophy regarding supporting both Python 2 and 3 is here: https://docs.djangoproject.com/en/dev/topics/python3/
Please give a read a let me know if you disagree with any of it so I don't head down the wrong path. I suspect there's no need to support Python 3.2, so we can probably omit the unicode_literals step, at least initially.

Supporting Python 3.3 or later seems to me like the way to go, given that 3.3 was released way back in 2012. 

It would be interesting to see which 3.x version of Python is shipping for each of the "compatible distros":

http://trac.edgewall.org/wiki/TracDev/ApiChanges/1.3#CompatibleDistros

- Ryan

[Greg Troxel]

RjOllos <rjo...@gmail.com> writes:

> Supporting Python 3.3 or later seems to me like the way to go, given that
> 3.3 was released way back in 2012.

That sounds reasonable

> It would be interesting to see which 3.x version of Python is shipping for
> each of the "compatible distros":
> http://trac.edgewall.org/wiki/TracDev/ApiChanges/1.3#CompatibleDistros

In pkgsrc (which isn't a "distro" ;-), python versions are 2.7, 3.3 and
3.4. Enough things don't work well with 3.x that I think it will be at
least a few years before it's reasonable to drop 2.7 support.

[Olemis Lang]

On 7/9/15, Greg Troxel <g...@ir.bbn.com> wrote:
>
> RjOllos <rjo...@gmail.com> writes:
>
>> Supporting Python 3.3 or later seems to me like the way to go, given that
>>
>> 3.3 was released way back in 2012.
>
> That sounds reasonable
>
>> It would be interesting to see which 3.x version of Python is shipping for
>>
>> each of the "compatible distros":
>> http://trac.edgewall.org/wiki/TracDev/ApiChanges/1.3#CompatibleDistros
>
> In pkgsrc (which isn't a "distro" ;-), python versions are 2.7, 3.3 and
> 3.4.

I'd suggest to consider 3.4 as the explicitly supported version so
that it would be possible to use async support [1]_ without having to
rely upon ugly hacks .

> Enough things don't work well with 3.x that I think it will be at
> least a few years before it's reasonable to drop 2.7 support.
>

+1

.. [1] https://www.python.org/dev/peps/pep-0492/

--
Regards,

Olemis - @olemislc

Apache™ Bloodhound contributor
http://issues.apache.org/bloodhound
http://blood-hound.net

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

[Tim Graham]

I put together a few initial patches (some is rebased from Jun's work) which should be fine to merge now, even if Python 2.6 is still supported.

Is attaching patches to Trac tickets the best way to submit this work? That workflow seems painful from a code review perspective.

I created a pull request to my own fork which seems like it would at least make review easier: https://github.com/timgraham/trac/pull/1

[Ryan Ollos]

On Sun, Jul 19, 2015 at 10:56 AM, Tim Graham <timog...@gmail.com> wrote:

I put together a few initial patches (some is rebased from Jun's work) which should be fine to merge now, even if Python 2.6 is still supported.

Is attaching patches to Trac tickets the best way to submit this work? That workflow seems painful from a code review perspective.

I created a pull request to my own fork which seems like it would at least make review easier: https://github.com/timgraham/trac/pull/1

Providing a link to Git repository in a ticket is good.  Looks like you have a good granularity to those changesets, which should make them easy to review. Thanks!

I think it would be good to just create another ticket for Python3 compatibility, one that we can close at the end of this development iteration even if we haven't obtained full Python3 compatibility. You can then post comments about your work to that ticket, and we can integrate it to the trunk.

When we finally have full Python3 compatibility we can close out #10083.

- Ryan

[RjOllos]

On Thursday, July 9, 2015 at 4:26:37 PM UTC-7, Greg Troxel wrote:

In pkgsrc (which isn't a "distro" ;-), python versions are 2.7, 3.3 and
3.4.  Enough things don't work well with 3.x that I think it will be at
least a few years before it's reasonable to drop 2.7 support.

Nothing of this is certain, but at current development pace and assuming that support for one Python version is dropped in every release:

Trac 1.4 will be released around July 2016 and drop support for Python 2.6

Trac 1.6 will be released around July 2017 and drop support for Python 2.7

and of course we will continue to support a Python-2.7-compatible Trac version as an LTS release for a while beyond 2017.

[Tim Graham]

Okay, created http://trac.edgewall.org/ticket/12130

I'll direct all future discussion there unless there's a reason to come back to the mailing list.

So that we don't need to rework a bunch of imports, it would be great to have a decision ASAP about whether to add six as a dependency or to vendor a copy of it. Currently I have it as a dependency.

[RjOllos]

On Tuesday, July 7, 2015 at 5:36:19 PM UTC-7, Tim Graham wrote:

By the way, Django's philosophy regarding supporting both Python 2 and 3 is here: https://docs.djangoproject.com/en/dev/topics/python3/
Please give a read a let me know if you disagree with any of it so I don't head down the wrong path. I suspect there's no need to support Python 3.2, so we can probably omit the unicode_literals step, at least initially.

I added a link to the Django document in your CodingStyle guide:

http://trac.edgewall.org/wiki/TracDev/CodingStyle?action=diff&version=31

I've given it a cursory read, but haven't spent much time on 2/3 porting. If anyone has concerns about items from the Django style guide we can discuss on the mailing list.

- Ryan