On Fri, Dec 2, 2011 at 9:55 AM, Vinay Sajip <vinay_sa...@yahoo.co.uk> wrote: > The Python 3 port now has all tests passing on 2.7.2 and 3.2.2 with > the same codebase:
WOOOO!
This is really fantastic news - I can't thank you enough for pushing this. I've just started scratching the surface and don't have any specific feedback yet, but I want to make sure you know how appreciative I am.
I'll try to spend the weekend writing/porting an app or two. If I can, I'll let you know how it goes.
So, last time I saw, you had only run against sqlite. Do you need help testing it against postgres, MySQL, and oracle (perhaps some unofficial ones)? What about all the caching backends? Do we need tests Python 3 equivalent tests for the ones that were skipped due to being Python 2-isms? What about some sort of "porting guide" or other needed documentation?
Basically, what's left, besides getting it merged into the official trunk that people can help you with?
>> The Python 3 port now has all tests passing on 2.7.2 and 3.2.2 with >> the same codebase:
> Wohoo! Can't wait to try py3 with Django! :)
> Regards, > Håkon Erichsen
> -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en.
On Fri, Dec 2, 2011 at 11:34 AM, Joe & Anne Tennies <tenn...@gmail.com> wrote:
> So, last time I saw, you had only run against sqlite. Do you need help > testing it against postgres, MySQL, and oracle (perhaps some unofficial > ones)? What about all the caching backends? Do we need tests Python 3 > equivalent tests for the ones that were skipped due to being Python 2-isms? > What about some sort of "porting guide" or other needed documentation?
> Basically, what's left, besides getting it merged into the official trunk > that people can help you with?
I'm already planning to go through the Oracle tests this weekend to get them ready for 1.4, so I'll run this through as well and let you know how it goes. Right now I expect failures even in 2.7.
On Dec 2, 6:34 pm, "Joe & Anne Tennies" <tenn...@gmail.com> wrote:
> So, last time I saw, you had only run against sqlite. Do you need help > testing it against postgres, MySQL, and oracle (perhaps some unofficial > ones)? What about all the caching backends? Do we need tests Python 3 > equivalent tests for the ones that were skipped due to being Python 2-isms? > What about some sort of "porting guide" or other needed documentation?
> Basically, what's left, besides getting it merged into the official trunk > that people can help you with?
Yes, you can help with one or all of the above items. As far as I know, the additional tests that are skipped are because of dependencies on PIL and setuptools (neither of which I have installed to run with Python 3: and there are some PIL ports for Python3, plus one can use distribute in place of setuptools. This is the area I am working on currently - working in a virtualenv with distribute and a PIL port installed.
You can certainly try helping with a PostgreSQL backend, I believe py- postgresql can be used as a PostgreSQL driver under Python 3. Ian Kelly mentioned on this thread that he'll be looking at the Oracle backend.
On Dec 2, 9:29 pm, Ian Kelly <ian.g.ke...@gmail.com> wrote:
> I'm already planning to go through the Oracle tests this weekend to > get them ready for 1.4, so I'll run this through as well and let you > know how it goes. Right now I expect failures even in 2.7.
Great, I look forward to the feedback. I can't do any testing at the moment with an Oracle backend, unfortunately - but I can certainly look at test failures under 3.x which succeed under 2.x to try and eyeball fixes to problems.
To follow up some discussion on reddit, the subject of exceptions was brought up. The Python 2.X/3.X compatible way to do exceptions requires:
e = sys.exc_info()[1]
...if you need the actual exception object. As discussed on reddit, this is slow on PyPy.
I did some checks in the py3k patch of all the cases where we actually do this because we need the exception object. I found the following:
= Cases where slowness probably doesn't matter too much =
== Developer errors ==
- template syntax errors and other errors
- incorrect configuration e.g. import errors due to incorrect dotted to path to some component
These are generally only observed during development, not production.
== Tests ==
- running tests - many instances at different levels of code - in unittest and in tests themselves.
It would be nice if test suite ran fast, but not so important.
== Runtime 'exceptional' cases ==
- file handling errors e.g. file already exists, permission errors. (These are likely 'rare' cases or misconfiguration, but conceivably they could be common).
- general view exception handling.
If any exception happens in a view, (including Http404), we pass it to the exception handling middleware. This also applies to decorators created from middleware. (So, this might not be such an 'exceptional' case).
- crashers of various kinds, especially in management commands e.g. can't serialize data in dumpdata command
- invalid HTTP requests
- some DB operational/integrity errors (the exception object is needed to map between the different types of errors that should be raised)
- most of these were already calling sys.exc_info()
= Cases where slowness probably does matter =
- signals - errors are caught and appended to 'responses'. Signals can be used for all kinds of things.
- wrapping DoesNotExist in IndexError in queryset slicing
- we could potentially change this behaviour - do we really need to capture the underlying exception arguments and pass to IndexError?
- Validation error handling - the validators in django.core.validators for URLs and email validation use an actual exception object.
- model validation and form validation - use exceptions to pass info around.
- errors that happen in templates but are silenced. (we check the actual exception object to see if we should silence them). Since they are silent, there could be any number of these going on in an app.
- MultiJoin in query generation code - don't know how often this is thrown and caught. We could conceivably change the way this works, it is internal.
- Resolver404 errors - very common, since it is part of the URL resolving mechanism. This is a documented API, so we can't change it.
I also found a couple of places where we can avoid creating the exception object if `settings.DEBUG = False` - untested patch attached.
Regards,
Luke
-- The fashion wears out more apparel than the man. -- William Shakespeare
On Dec 3, 8:50 pm, Karen Tracey <kmtra...@gmail.com> wrote:
> On Fri, Dec 2, 2011 at 10:55 AM, Vinay Sajip <vinay_sa...@yahoo.co.uk>wrote:
> > The Python 3 port now has all tests passing on 2.7.2 and 3.2.2 with > > the same codebase:
> What about Pythons 2.5 and 2.6?
I've done nothing intentionally that prevents the port from working on 2.6 and 2.7 - it's just that my test machine happens to be a recent Ubuntu variant that doesn't have them installed.
On Dec 3, 8:50 pm, Karen Tracey <kmtra...@gmail.com> wrote:
> What about Pythons 2.5 and 2.6?
I did a bit more checking, and a bit more work will be required on 2.5 and 2.6: for example, on 2.5, parse_qsl needs to come from elsewhere, and on 2.6, unittest.skipIf needs to come from somewhere else. So some changes need to be made to django/utils/py3.py to accommodate this - it shouldn't be too much work, hopefully :-)
On Dec 3, 8:50 pm, Karen Tracey <kmtra...@gmail.com> wrote:
> What about Pythons 2.5 and 2.6?
Okay, I've now done some more testing, and with some minor changes, I can report that good results were obtained on Pythons 2.5.4 and 2.6.2. The remaining failures are representational ones - u'foo' vs. 'foo' and order of dictionary keys in doctest output. Summaries follow.
On Dec 3, 9:18 pm, Luke Plant <L.Plant...@cantab.net> wrote:
> I did some checks in the py3k patch of all the cases where we actually > do this because we need the exception object. I found the following:
Thanks for the analysis and suggested patch. I've implemented this patch in my branch, and the tests are running now on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 :-)
I've just finished going through the oracle tests and sent you a pull request that fixes a few issues:
* compiler.py still had a map(None) call, that I replaced with izip_longest. * cx_Oracle doesn't seem to want to accept bools as bind parameters with Python 3; these had to be converted to ints. * we were calling iterator.next() in one place instead of next(iterator) * a couple of tests (one of which is specifically an Oracle test) needed minor updates
The biggest change was just adding izip_longest to utils.py3 and utils.itercompat.
With these changes, I'm not seeing any failures with oracle that I'm not also seeing with sqlite3. I am seeing a bunch of failures, though.
Most of the failures are coming from the timezones tests.
Also, one rather obnoxious failure that I encountered was the file_uploads.FileUploadTests.test_large_upload test, which doesn't actually fail but just hangs indefinitely. I had to skip the test in order to run the full suite.
On Tuesday, December 6, 2011, Ian Kelly <ian.g.ke...@gmail.com> wrote: > itertools.izip_longest was added in Python 2.6, though, so a > compatibility function is needed for Python 2.5.
Ahh, that's a good catch. I'll have to make use of your solution, then.
On Dec 7, 5:53 am, Ian Clelland <clell...@gmail.com> wrote:
> On Tuesday, December 6, 2011, Ian Kelly <ian.g.ke...@gmail.com> wrote: > > itertools.izip_longest was added in Python 2.6, though, so a > > compatibility function is needed for Python 2.5.
> Ahh, that's a good catch. I'll have to make use of your solution, then.
I've already pulled Ian Kelly's changes from his BitBucket repo. Which MySQL driver are you using for Python 3? I started looking at
git://github.com/davispuh/MySQL-for-Python-3.git
but it appears to expect queries formatted with {} rather than %s, so it doesn't seem that I can use it.
