Porting web.py to Python 3

[Mathieu Xhonneux]

Hi,

I've been hired by the CS departement of the Université Catholique de Louvain (UCL, Belgium) to port web.py to Python 3. We're using the framework in some of our internal tools and we do need a Py3 upgrade for them.

I've already done some work, available here : https://github.com/Zashas/webpy/tree/py3

We saw that Anand began the migration 3 years ago and decided to pursue his work (making the codebase compatible with Py2 and Py3, which seems to me the best option, btw Django manages it successfully). I integrated the commits he already wrote back then, ported all the unittests to Py3 and now I'm busy making them pass in both versions of Python.

The ORM seems to be OK (I'm kinda hoping the tests' coverage is wide enough), but it originally supports some DB backends that are either no longer maintained, or not ported to Py3 :

• psycopg1 : evolved to psycopg2, I'm guessing that not much people are using it these days
  
• pysqlite2 : not ported to Py3
• MySQLdb : not ported to Py3, but mysqlclient seems to be compatible and to do the job fairly enough
• DBUtils : not ported to Py3

Maybe we should consider dropping support for these ? Their presence is not doing any harm in the codebase, but I don't think that psycopg1 and pysqlite2 are still useful, and removing them could clean the code a bit. 

Anyway, if anyone wants to follow my work, I'm open to any remark, suggestion or patch. I'm hoping to have ported everything by friday, but it will definitively require some testing.

Mathieu

 

[qyb]

In my opinion, DBUtils' connection pool is a important feature.

--
You received this message because you are subscribed to the Google Groups "web.py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webpy+un...@googlegroups.com.
To post to this group, send email to we...@googlegroups.com.
Visit this group at https://groups.google.com/group/webpy.
For more options, visit https://groups.google.com/d/optout.

[James Tyra]

First, thank you Mathieu for the work you are doing. Although people might not publicly say it, there are lots of us still using web.py in various places who would love a Py3 version.

As to the actual question, I would put all of the DB related things as a very low priority. There is a lovely python db module entitled "Peewee" - http://docs.peewee-orm.com/en/latest/ , which very much fits right in with web.py (in spirit and technically).  I would much rather see us push web.py py3 users to this module (which is actively maintained and updated) and drop the current DB support. 

Peewee has connection pooling, support for Sqlite, MySQL, Postgres and others. It is extremely easy to use, and supports modern features. 

I've been using web.py and Peewee on a medium traffic site with great results for the past 1.5 years now. It has been rock solid and the way forward In my opinion.

-James

[Anand Chitipothu]

On Tue, 21 Jun 2016 at 14:48 Mathieu Xhonneux <m.xho...@gmail.com> wrote:

Hi,

I've been hired by the CS departement of the Université Catholique de Louvain (UCL, Belgium) to port web.py to Python 3. We're using the framework in some of our internal tools and we do need a Py3 upgrade for them.

Thats a great news! And thank you for taking up this task.

 

I've already done some work, available here : https://github.com/Zashas/webpy/tree/py3

We saw that Anand began the migration 3 years ago and decided to pursue his work (making the codebase compatible with Py2 and Py3, which seems to me the best option, btw Django manages it successfully). I integrated the commits he already wrote back then, ported all the unittests to Py3 and now I'm busy making them pass in both versions of Python.

I'll review it in next couple of days and lets try to merge into master as soon as possible. 

The current version of python maintains compatibility with Python 2.4 onwards. I think we can drop 2.4 and 2.5 completely. I'm in favour of dropping 2.6 as well and retain only 2.7 and Python 3.5+.

 

The ORM seems to be OK (I'm kinda hoping the tests' coverage is wide enough), but it originally supports some DB backends that are either no longer maintained, or not ported to Py3 :

• psycopg1 : evolved to psycopg2, I'm guessing that not much people are using it these days
  
• pysqlite2 : not ported to Py3
• MySQLdb : not ported to Py3, but mysqlclient seems to be compatible and to do the job fairly enough
• DBUtils : not ported to Py3

Maybe we should consider dropping support for these ? Their presence is not doing any harm in the codebase, but I don't think that psycopg1 and pysqlite2 are still useful, and removing them could clean the code a bit. 

I think db module is very important piece. Yes, we can drop all those legacy postgres modules. Switching to mysqlclient for Python3 seems likes a good idea to me.

It is desirable to have connection polling. If DBUtils doesn't work with Python 3, lets try to find alternative.

 

Anyway, if anyone wants to follow my work, I'm open to any remark, suggestion or patch. I'm hoping to have ported everything by friday, but it will definitively require some testing.

Test coverage is really poor for web.py. Good to have that fixed as we are making lot of changes. Any one interested to help with that?

Anand

[Kenny Rachuonyo]

Hello Anand,

What would be the starting point to identify the parts that need more
test coverage?

> --
> You received this message because you are subscribed to the Google Groups
> "web.py" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to webpy+un...@googlegroups.com.
> To post to this group, send email to we...@googlegroups.com.
> Visit this group at https://groups.google.com/group/webpy.
> For more options, visit https://groups.google.com/d/optout.



--

"The difference between theory and practice tends to be very small in
theory, but in practice it is very large indeed."

- Anon

www.99nth.com/~krm

[Mathieu Xhonneux]

Since I'm browsing all the codebase for the portage, I'll make a list of sections needing tests. From what I saw until now, web/session.py could use some doctests.

[Sathish S]

Hi,

I hope this link may be useful https://github.com/dawn110110/templator

-Sathish

[Mathieu Xhonneux]

The webpy test coverage is actually not that bad. I've run coverage.py across the codebase (in my Py2/Py3 branch) and I got a nice 73%, details can be seen here : http://mathx.alwaysdata.net/webpycov/

Most of the uncovered code concerns Py3-specific code, exceptions, little corner cases and unused features (some functions could be removed from utils.py for instance).

[Mathieu Xhonneux]

I've finished the porting, PR is available here : https://github.com/webpy/webpy/pull/370

[Anand Chitipothu]

On Fri, 24 Jun 2016 at 16:28 Mathieu Xhonneux <mat...@gmail.com> wrote:

I've finished the porting, PR is available here : https://github.com/webpy/webpy/pull/370

Awesome! I'll review it and get back to you tonight.

Anand

[Anand Chitipothu]

I've pushed the py3 branch to webpy repo and made couple of more changes. 

https://github.com/webpy/webpy/tree/py3

Found couple of issues. 

https://github.com/webpy/webpy/issues?q=is%3Aissue+is%3Aopen+label%3Apy3

If anyone of you is interested to try fixing it, you are welcome. 

Please try your applications with this branch and report any issues that you find. Mathieu has done an amazing job and I would love to see that released soon.

Anand

[Anand Chitipothu]

By the way, all the tests are passing in the py3 branch.

https://travis-ci.org/webpy/webpy/branches

Anand

[Zhang Huangbin]

On Wednesday, June 22, 2016 at 1:12:51 AM UTC+8, Anand wrote:

The current version of python maintains compatibility with Python 2.4 onwards. I think we can drop 2.4 and 2.5 completely. I'm in favour of dropping 2.6 as well and retain only 2.7 and Python 3.5+.

*) RHEL/CentOS 6 ships Python-2.6, and still widely used by many users. So it might be great to have py26 support in web.py.

*) Another wish is, supports database connection pool (with DBUtils or other alternatives).

So happy to see more people step in. Thank you. :)

[tensio]

We should drop 2.6 as well. 

Look at the Python Version Support cycles[1]. 2.6 is not maintained for 3 years now. 

So while 2.7's clock is ticking we should focus on 3.x support. 

[1]: https://docs.python.org/devguide/#status-of-python-branches