Porting Genshi to Python 3

3 views
Skip to first unread message

Simon Cross

unread,
Jul 19, 2010, 1:18:43 PM7/19/10
to gen...@googlegroups.com
Greetings

The Cape Town Python Users Group is considering holding a sprint to
port Genshi to Python 3 in early September.

Before I start throwing any real time at organizing this, some quick questions:

* Anyone else worked on or planning to work on porting Genshi to
Python 3 already?
* Any objections?
* Any suggestions on preferred approach?

Ideally we'll end up with a single code base that works under both 2.x
and 3.x but otherwise we'll fall back to creating something that works
with 2to3. Initially we'll skip porting the C stuff and tackle it
later on in the sprint if things go well. The sprint commits can
happen on a DVCS copy of Genshi trunk and then we can hopefully get a
commiter to pull them across once everyone is happy with the changes.

Thoughts welcome.

Schiavo
Simon

Jeroen Ruigrok van der Werven

unread,
Jul 20, 2010, 4:49:13 AM7/20/10
to gen...@googlegroups.com
Hi Simon,

-On [20100719 19:19], Simon Cross (hodg...@gmail.com) wrote:
>The Cape Town Python Users Group is considering holding a sprint to
>port Genshi to Python 3 in early September.

Quite cool.

>* Anyone else worked on or planning to work on porting Genshi to
>Python 3 already?

Not me, I am trying to free time to work on Babel. And I am not sure where
Chris is currently.

>* Any objections?

Not from me.

>* Any suggestions on preferred approach?

I do remember Chris saying that he wanted to do an automatic 2to3 pass over
two separate trees.

>Ideally we'll end up with a single code base that works under both 2.x
>and 3.x but otherwise we'll fall back to creating something that works
>with 2to3.

Distribute has some additional functionality for setup.py to automatically
apply 2to3. Might be useful as a starting point.

>Initially we'll skip porting the C stuff and tackle it
>later on in the sprint if things go well. The sprint commits can
>happen on a DVCS copy of Genshi trunk and then we can hopefully get a
>commiter to pull them across once everyone is happy with the changes.

I can always create an official branch to put any results of the group in.

--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Religion... Is the opium of the people...

Simon Cross

unread,
Jul 20, 2010, 5:30:05 AM7/20/10
to gen...@googlegroups.com
2010/7/20 Jeroen Ruigrok van der Werven <asm...@in-nomine.org>:

> I do remember Chris saying that he wanted to do an automatic 2to3 pass over
> two separate trees.

Does this mean trunk and a tree tweaked to play nicely with 2to3?

> Distribute has some additional functionality for setup.py to automatically
> apply 2to3. Might be useful as a starting point.

I'm aware of this. Will be useful if we have to rely on 2to3. I'd like
to avoid adding any new "dependencies" for installing Genshi if
possible though.

> I can always create an official branch to put any results of the group in.

That would be cool. We may still using a DVCS to play around in but it
would be nice to have somewhere official to put the result.

Schiavo
Simon

Jeroen Ruigrok van der Werven

unread,
Jul 20, 2010, 6:27:43 AM7/20/10
to gen...@googlegroups.com
-On [20100720 11:30], Simon Cross (hodg...@gmail.com) wrote:
>2010/7/20 Jeroen Ruigrok van der Werven <asm...@in-nomine.org>:
>> I do remember Chris saying that he wanted to do an automatic 2to3 pass over
>> two separate trees.
>
>Does this mean trunk and a tree tweaked to play nicely with 2to3?

Crap, sorry, mixed what I wanted to say, rephrase:

I do remember Chris saying he wanted to do an automatic 2to3 pass over the
code instead of maintaining two separate trees.

Mea culpa.

[Distribute and 2to3]


>I'm aware of this. Will be useful if we have to rely on 2to3. I'd like
>to avoid adding any new "dependencies" for installing Genshi if
>possible though.

I think it's an option you can use to automatically run 2to3 but ignored on
others. I saw that SQLAlchemy also has conversion hooks in its setup.py
system, might be worthwhile to look for inspiration there too.

>> I can always create an official branch to put any results of the group in.
>
>That would be cool. We may still using a DVCS to play around in but it
>would be nice to have somewhere official to put the result.

Official as in I have commit rights. I don't trust myself with the internals
of Genshi though. But hey, we have testsuites for that. ^^

--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B

Unto the pure all things are pure...

Simon Cross

unread,
Jul 20, 2010, 2:17:43 PM7/20/10
to gen...@googlegroups.com
2010/7/20 Jeroen Ruigrok van der Werven <asm...@in-nomine.org>:
> Official as in I have commit rights. I don't trust myself with the internals
> of Genshi though. But hey, we have testsuites for that. ^^

If the changes are not too scary and the test suite passes we could
rely on people using trunk to test them before the next Genshi
release?

It shouldn't be too hard to find real world applications to test that
everything still works under 2.x but it might be a little more tricky
to find something substantial to test that really applications still
work under 3.x (since presumably there aren't any right now).

Schiavo
Simon

Jeroen Ruigrok van der Werven

unread,
Jul 20, 2010, 3:39:19 PM7/20/10
to gen...@googlegroups.com
-On [20100720 20:18], Simon Cross (hodg...@gmail.com) wrote:
>If the changes are not too scary and the test suite passes we could
>rely on people using trunk to test them before the next Genshi
>release?

I'm all for it, I will try to first contact Chris about it though, but if he
doesn't respond in a week or 3-4 I will just commit it (that'll be sure to
get his attention ;) ).

>It shouldn't be too hard to find real world applications to test that
>everything still works under 2.x but it might be a little more tricky
>to find something substantial to test that really applications still
>work under 3.x (since presumably there aren't any right now).

Not unless someone would port Trac. ^^
My own pet project has been rusting over the months, so it would require a
lot of updating first before I could even think of testing. But maybe some
list members are toying with Python 3 as well and could help out.

--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B

All the pain I caused myself was out of love for others...

Joshua Rowley

unread,
Jul 22, 2010, 5:24:58 AM7/22/10
to gen...@googlegroups.com
We'll be using Genshi in our Py3k framework, but it's a work in progress at this stage.

Our useage will be quite demanding, however, so there will be a lot of testing going on.

Josh

2010/7/20 Jeroen Ruigrok van der Werven <asm...@in-nomine.org>

--
You received this message because you are subscribed to the Google Groups "Genshi" group.
To post to this group, send email to gen...@googlegroups.com.
To unsubscribe from this group, send email to genshi+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/genshi?hl=en.


Simon Cross

unread,
Jul 22, 2010, 5:34:20 AM7/22/10
to gen...@googlegroups.com
2010/7/22 Joshua Rowley <jos...@vannelluna.com>:

> We'll be using Genshi in our Py3k framework, but it's a work in progress at
> this stage.
>
> Our useage will be quite demanding, however, so there will be a lot of
> testing going on.

It would be really cool to get some early testing of our port if
possible. If you're around on the weekend of the sprint you're
welcome to join us on our IRC channel [1].

[1] See http://ctpug.org.za/

Schiavo
Simon

Graham Poulter

unread,
Aug 31, 2010, 3:19:25 AM8/31/10
to Genshi
Hello,

I have a bit of personal experience to share on porting code to Python
3, and I'll be at the Cape Town Python Users Group for the Genshi
porting sprint.

On Jul 20, 12:27 pm, Jeroen Ruigrok van der Werven <asmo...@in-
nomine.org> wrote:
> -On [20100720 11:30], Simon Cross (hodges...@gmail.com) wrote:
>
> >2010/7/20 Jeroen Ruigrok van der Werven <asmo...@in-nomine.org>:
> >> I do remember Chris saying that he wanted to do an automatic 2to3 pass over
> >> two separate trees.
>
> >Does this mean trunk and a tree tweaked to play nicely with 2to3?
>
> Crap, sorry, mixed what I wanted to say, rephrase:
>
> I do remember Chris saying he wanted to do an automatic 2to3 pass over the
> code instead of maintaining two separate trees.
>
> Mea culpa.

Single code base with automatic 2to3 conversion is indeed the
recommended approach in then Python 3.0 release notes:

"If you are maintaining a library that needs to support both Python
2.6 and Python 3.0, the best approach is to modify step 3 above by
editing the 2.6 version of the source code and running the 2to3
translator again, rather than editing the 3.0 version of the source
code."

I used the 2to3 approach on a small library (python-ngram) and found
it easy to modify Python 2.x code so that it converts cleanly, by
iterating until 2to3 gave no more errors. I haven't ported C
extensions, but Google shows a "2to3c" which may be worth a look as a
starting point for keeping a 3.x branch of the C module. There is
also a "3to2.py" (http://wiki.python.org/moin/3to2), so one day in the
far future when most people are running Py3K it will be possible to
run "2to3" one last time and switch over to using 3to2.py to back-port
the library to Python 2.

> [Distribute and 2to3]
>
> >I'm aware of this. Will be useful if we have to rely on 2to3. I'd like
> >to avoid adding any new "dependencies" for installing Genshi if
> >possible though.
>
> I think it's an option you can use to automatically run 2to3 but ignored on
> others. I saw that SQLAlchemy also has conversion hooks in its setup.py
> system, might be worthwhile to look for inspiration there too.

I've also used distribute with 2to3 (http://packages.python.org/
distribute/python3.html), by adding a check for Python 3 that adds
"2to3" to the setup() arguments, so that setup.py remains compatible
with distutils/setuptools under Python 2.x. However, the 2to3 step
does slow down installation under Python 3, and I needto remember to
test 3.x installation after I've made changes to the 2.x code.

-- Graham
Reply all
Reply to author
Forward
0 new messages