[Django] #25875: Unicode error when converting a Q object to a string
Django
----------------------------------------------+----------------------------
Reporter: benjaminjkraft | Owner: benjaminjkraft
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+----------------------------
If you have a model whose `__str__` returns non-ASCII text, and you refer
to that model in a `Q` object, attempting to `print` that `Q` object, or
otherwise convert it to a string, will error. This is a regression in
Django 1.5, caused indirectly by
[https://github.com/django/django/commit/3dd5d72 3dd5d72], and remains an
issue in master; I believe it should only affect python 2 but I haven't
actually checked.
The issue appears to be that `django.utils.tree.Node.__str__`
inadvertently attempts to convert the string back to unicode. In
particular, since [https://github.com/django/django/commit/3dd5d72
3dd5d72], `self.connector` is a unicode string if `self` is a `Q` object.
This causes python to convert the string being formatted and all its
formatting arguments to unicode. Since this is done implicitly it uses
the default encoding (ASCII), which fails.
Fixing this should be as simple as just explicitly doing
`str(self.connector)` instead, although I haven't yet run the tests to see
if that breaks anything else. In any case, I'm happy to write a patch to
do that later this week. It's possible it would be better to instead
define a `__unicode__` method and avoid the string-conversion entirely,
I'm not sure, but this seems simpler.
Complete repro:
{{{
# in models.py
class Foo(models.Model):
name = models.TextField()
def __unicode__(self):
return u'Foö'
def __str__(self):
return unicode(self).encode('utf-8')
def __repr__(self):
return str(self)
# in a shell
Q(baz=Foo(name="baz")) # raises UnicodeDecodeError
}}}
Note that the repro works as long as at least one of `__unicode__`,
`__str__`, and `__repr__` is defined and returns non-ASCII text -- they
don't need to all be explicitly defined.
--
Ticket URL: <https://code.djangoproject.com/ticket/25875>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: benjaminjkraft | Owner:
| benjaminjkraft
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by jarshwah):
* needs_docs: => 0
* needs_better_patch: => 0
* needs_tests: => 0
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/25875#comment:1>
Django
Reporter: benjaminjkraft | Owner:
| benjaminjkraft
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/25875#comment:2>
Django
Reporter: benjaminjkraft | Owner:
| benjaminjkraft
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by charettes):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/25875#comment:3>
Django
Reporter: benjaminjkraft | Owner:
| benjaminjkraft
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Claude Paroz <claude@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"ed20dd2e85a0bbd45085809417c29e92ced5e618" ed20dd2e]:
{{{
#!CommitTicketReference repository=""
revision="ed20dd2e85a0bbd45085809417c29e92ced5e618"
Fixed #25875 -- Prevented UnicodeDecodeError for Q object repr
Thanks Ben Kraft for the report, and Simon Charette for the review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25875#comment:4>