Django on Python 2.6

[Karen Tracey]

A beta 2 release of Python 2.6 was released yesterday or so, and they are on a schedule to have a final release around the same time Django 1.0 is scheduled.  Actually, slightly after, but who knows with scheduling code releases which one will actually reach 'final' first. 

Right now, http://www.djangoproject.com/documentation/install/ says Django "works with any Python version 2.3 and higher".  I expect that to change when Python 3.0 comes out to say something more like "works with Python 2.x where x >= 3", since I know Django will not likely work with Python 3.0 as soon as it comes out.  However, I expect 2.6 will be as supported by Django 1.0 as 2.3, 2.4, and 2.5, right?

And it should Just Work, since 2.6 is supposed to be backwards compatible, but you know how that goes.  So today I experimented with running the Django test suite on Python 2.6b2 to see how close it is to just working.  Not surprisingly, I ran into a few problems. 

First, there are some newly-deprecated modules like md5 and sha.  Probably the test suite should be updated to try hashlib and only fall back to these others if necessary?

Second theres a reappearance of a Python bug that was was present in 2.3 but gone in 2.4/2.5.  I put a note in the bug (http://bugs.python.org/issue1242657) and it's been reopened on the Python side.

Third the repr of Decimal has changed to use double quotes instead of single quotes, so every test that checks for a Decimal value fails.  This strikes me as a very odd (and backwards-incompatible) change to have made and I've asked about it on the Python list (users, not dev, for starters) but I'd be interested in what other people think of this change.

Fourth, there are several failures where sqlite (the only backend I have for 2.6) complains about being handed 8-bit bytestrings instead of Unicode:

        return Database.Cursor.execute(self, query, params)
    ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

This surprised me since I expected Django to be passing Unicode.  But some tests, in fact, use simple bytestrings, for example:

http://code.djangoproject.com/browser/django/trunk/tests/modeltests/custom_pk/models.py#L107

has: >>> emp = Employee(employee_code='jaźń')

Shouldn't that be: >>> emp = Employee(employee_code=u'jaźń')

? 

Changing it like so makes the test succeed on 2.6b2 and has no apparent ill-effects on 2.5.  But I'm a little surprised this hasn't caused a problem before, so I fear I'm missing something here?

Fourth, fifth, six...there are a few more I haven't looked at in any detail yet.  I thought I'd first check and make sure this is worthwhile (Django will be expecting to support 2.6 when it comes out, right?) and get some feedback on the questions I've got above.

Karen

[Jacob Kaplan-Moss]

Hey Karen --

Thanks for looking into this. I don't have the bandwidth to think
about it right now, but we should make sure that Django 1.0 final
works with Python 2.6. Let's make this a target for the 1.0 beta2
release?

Jacob

[Karen Tracey]

OK, I will hopefully have a block of time to look some more at this today and open tickets with beta milestones for any changes that need to be made. 

Karen

[Karen Tracey]

Following up on Django on Python 2.6, the tests are doing much better.  On my own machines (Linux & Windows) running Python 2.6 beta2 the failures I see now are:

1 - modeltests/model_forms failing on an expectation of getting ImageFileFiled instead of FileFiled.  When I first noticed this I also noticed there is already a ticket open for it (#8196) so I don't belive it is Python 2.6 related, rather it's because I don't have PIL?  I'm assuming someone will fix #8196 and this will go away, therefore I've not investigated it any further.

2 - regressiontests/queries fails on an expectation of getting an exception "FieldError: Infinite loop caused by ordering." instead of an empty list.  This is Python issue #1242657, a bug present in 2.3, fixed in 2.4/2.5 and reappearing in 2.6.  If it's not fixed in Python 2.6 before Django 1.0 I suppose we could always just skip those tests for 2.6 (as we do for 2.3)?  Effect is when you specify your models to have an invalid ordering, instead of an error pointing out where you went wrong, you get empty results.  Probably puzzling and hard to track down but ultimately it's an error in the Django app.

3 - on Linux only I get deprecation warnings from use of popen3.  It's used by the admin_scripts tests.  I've got a patch that uses subprocess instead and seems to work correctly on Windows (Python 2.3.5, 2.5.1, 2.6b2) and Linux (Python 2.5.1, 2.6b2).  I haven't written a ticket up yet because I've been short on time and wanted to try to understand why it's only deprecated on Linux and if using subprocess on Windows (where it's not being reported as deprecated) has any ill-effects.  If anyone has any insight here please speak up.

The Python community buildbot (http://www.python.org/dev/buildbot/community/all/?show=sparc%20Solaris%2010%20trunk) sees these errors plus a couple-three more:

4 - Error importing gis due to missing _ctypes. This also has a ticket open for it (#8207) so doesn't seem to be Python 2.6 related.  (Not sure why I'm not seeing this on my machines.)

5 - Another case of expected output not matching what's gotten in regressiontests/queries, this one having to do with ordering.  Haven't had time to track this down at all yet, so I don't know if it's due to the buildbot machine running current 2.6 trunk code vs. my machines using beta2 or some other difference in it vs. my machines.

6 - regressiontests/forms failing on file ouptut paths not maching.  Again I haven't yet had time to look at this closely. 

Wow, that sounds like a lot but it's actually way better than it was at first.  Thanks to the core devs for giving feedback on how to fix things and checking in patches and Martin v. Löwis for providing a patch for one of the more puzzling failures.

Karen

[Malcolm Tredinnick]

On Sun, 2008-08-10 at 12:09 -0400, Karen Tracey wrote:
> Following up on Django on Python 2.6, the tests are doing much better.
> On my own machines (Linux & Windows) running Python 2.6 beta2 the
> failures I see now are:

[... snip...]

This looks pretty good. A few known bugs on our end. One known bug on
Python's end (used to be two, but Martin's fix of the PySQLite thing
means we work around that one now). So they'll all be fixed or we can
work around them with a clean conscience.

Two of the items are really errors in the test suite, rather than errors
in Django or Python. We can fix those as time permits...

> 5 - Another case of expected output not matching what's gotten in
> regressiontests/queries, this one having to do with ordering. Haven't
> had time to track this down at all yet, so I don't know if it's due to
> the buildbot machine running current 2.6 trunk code vs. my machines
> using beta2 or some other difference in it vs. my machines.

I can rewrite that test to order on something more predictable. It's
making a bad assumption as written (assuming pk values will be
increasing as we insert), so fixing it properly is the right solution
there.

> 6 - regressiontests/forms failing on file ouptut paths not maching.
> Again I haven't yet had time to look at this closely.

That test looks fragile on our end, regardless of the result. If I get
time I might have a poke at finding a better way of writing that. It's a
bit too sensitive to whatever happens to be in a particular source
directory, rather than creating a known environment to test.

Regards,
Malcolm