[Django] #27901: Problem with Django & MySQL & model fields returning custom unicode classes
Django
-------------------------------------+-------------------------------------
Reporter: Dominique | Owner: nobody
Barton |
Type: Bug | Status: new
Component: Database | Version: 1.10
layer (models, ORM) | Keywords: mysql, unicode,
Severity: Normal | error
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Hi there
I use a custom model field (inherited from `TextField`) which simply
returns its value as an "enhanced" `unicode` object.
Unfortunately, this will raise UnicodeErrors as MySQLdb apparently fails
recognizing those values as `unicode`. The comparision isn't very clever
there ;)
MySQLdb has a `use_unicode` flag, but I don't know how to implement it in
Django and I think this is a serious bug.
I've prepared a whole test case right here: https://github.com/domibarton
/django-mysql-unicode-error
I hope this is enough for a "proper" bug description.
Cheers
Domi
--
Ticket URL: <https://code.djangoproject.com/ticket/27901>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution: needsinfo
Keywords: mysql, unicode, | Triage Stage:
error | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => needsinfo
Comment:
In this case, you need to explain why Django is at fault and ideally
provide a patch that fixes the issue. From what you provided so far, it
looks like you're asking for help debugging your custom field. For ways to
get help with that, see TicketClosingReasons/UseSupportChannels.
--
Ticket URL: <https://code.djangoproject.com/ticket/27901#comment:1>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution: needsinfo
Keywords: mysql, unicode, | Triage Stage:
error | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Claude Paroz):
You might also read https://docs.djangoproject.com/en/stable/howto/custom-
model-fields/#converting-query-values-to-database-values
--
Ticket URL: <https://code.djangoproject.com/ticket/27901#comment:2>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution: needsinfo
Keywords: mysql, unicode, | Triage Stage:
error | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Dominique Barton):
I was in the IRC channels and they sent me here. I specifically asked them
"is this really OK to file a bug" and the answer was "yes it is".
Now you tell me I've to go back?
--
Ticket URL: <https://code.djangoproject.com/ticket/27901#comment:3>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution:
error | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: needsinfo =>
Comment:
Replying to [comment:1 Tim Graham]:
> From what you provided so far, it looks like you're asking for help
debugging your custom field.
No I don't need help with debugging my custom field as my custom field is
not the problem! As I described before, IT IS A BUG and therefore I think
this ticket is quite valid!
Let's keep it simple and ignore the fact that I'm trying to make a custom
model field. Use a stock `TextField`, write a value to it and then save
the model. Now do the same, but use a custom `unicode` class for the
value. Something like that:
https://gist.github.com/domibarton/aa691342321f63f3511bb1ff53b946c7
--
Ticket URL: <https://code.djangoproject.com/ticket/27901#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Dominique Barton | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution:
Keywords: mysql, unicode, | Triage Stage:
error | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> Hi there
>
> I use a custom model field (inherited from `TextField`) which simply
> returns its value as an "enhanced" `unicode` object.
> Unfortunately, this will raise UnicodeErrors as MySQLdb apparently fails
> recognizing those values as `unicode`. The comparision isn't very clever
> there ;)
>
> MySQLdb has a `use_unicode` flag, but I don't know how to implement it in
> Django and I think this is a serious bug.
>
> I've prepared a whole test case right here: https://github.com/domibarton
> /django-mysql-unicode-error
>
> I hope this is enough for a "proper" bug description.
>
> Cheers
> Domi
New description:
Hi there
I use a custom model field (inherited from `TextField`) which simply
returns its value as an "enhanced" `unicode` object.
Unfortunately, this will raise UnicodeErrors as MySQLdb apparently fails
recognizing those values as `unicode`. The comparision isn't very clever
there ;)
MySQLdb has a `use_unicode` flag, but I don't know how to implement it in
Django and I think this is a serious bug.
I've prepared a whole test case right here: https://github.com/domibarton
/django-mysql-unicode-error
I hope this is enough for a "proper" bug description.
Cheers
Domi
'''EDIT:''' It has NOTHING to do with my custom text field, this was just
an example. Basically Django (or MySQLdb) fails when writing non-`unicode`
objects (but `unicode`-inherited objects) values to a `TextField`. Proof
for that issue is right here:
https://gist.github.com/domibarton/aa691342321f63f3511bb1ff53b946c7
--
--
Ticket URL: <https://code.djangoproject.com/ticket/27901#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Dominique Barton | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution:
Keywords: mysql, unicode, | Triage Stage:
error | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Dominique Barton):
Additional info:
Switching from `mysql-python` to `mysqlcient` didn't help.
However, switching from python2 to python3 solves the issue.
This might be because in py3 everything is unicode by default (vs. byte
strings in py2) and the "default" type is unicode.
I'll try to migrate my project to py3 and the bug is gone. Unfortunately
it still exists with py2.
--
Ticket URL: <https://code.djangoproject.com/ticket/27901#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Dominique Barton | Owner: nobody
Component: Database layer | Version: 1.10
(models, ORM) |
Keywords: mysql, unicode, | Triage Stage:
error | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => wontfix
Comment:
Even if Django is at fault (it's still not clear to me if that's the
case), the time for fixing issues that only affect Python 2 has passed
since Django's master branch only supports Python 3.
--
Ticket URL: <https://code.djangoproject.com/ticket/27901#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Dominique Barton | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: mysql, unicode, | Triage Stage:
error | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Dominique Barton):
Replying to [comment:7 Tim Graham]:
> Even if Django is at fault (it's still not clear to me if that's the
case), the time for fixing issues that only affect Python 2 has passed
since Django's master branch only supports Python 3.
Didn't know that. OK, I'll try to move everything to py3.
--
Ticket URL: <https://code.djangoproject.com/ticket/27901#comment:8>