[Django] #21933: UnicodeDecodeError when writing unicode to stdout of management command
Django
-------------------------------+--------------------
Reporter: synotna | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.6
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------
In a management command, if you include unicode characters when writing to
stdout (with self.stdout.write) you will get a UnicodeDecodeError
This happens even if the system locale is utf-8, and self.stdout.encoding
is utf-8, and you .decode(self.stdout.encoding,'replace')
A workaround is to always .decode() to ascii, and if that is what should
be done it should be documented at
https://docs.djangoproject.com/en/dev/howto/custom-management-commands/
/myapp/management/commands/unicode_test.py
{{{
# -*- coding: utf-8 -*-
from django.core.management.base import BaseCommand
class Command(BaseCommand):
args = '<none>'
help = 'Do something'
def handle(self, *args, **options):
output = 'Σάββα▒^▒ο'
self.stdout.write('Output: {}'.format(output))
}}}
{{{
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/__init__.py", line 399, in
execute_from_command_line
utility.execute()
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/home/tech/esldj/all-in-
one/go4/management/commands/unicode_test.py", line 10, in handle
self.stdout.write('Output: {}'.format(output))
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/base.py", line 65, in write
if ending and not msg.endswith(ending):
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21933>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: synotna | Owner: nobody
Type: Uncategorized | Status: closed
Component: Uncategorized | Version: 1.6
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* status: new => closed
* needs_better_patch: => 0
* resolution: => invalid
* needs_tests: => 0
* needs_docs: => 0
Comment:
Your `output` attribute in the above code example should be prefixed by
`u`.
--
Ticket URL: <https://code.djangoproject.com/ticket/21933#comment:1>
Django
Reporter: synotna | Owner: nobody
Type: Uncategorized | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by dhobbs):
* status: closed => new
* version: 1.6 => 1.8
* resolution: invalid =>
Old description:
New description:
In a management command, if you include unicode characters when writing to
stdout (with self.stdout.write) you will get a UnicodeDecodeError
This happens even if the system locale is utf-8, and self.stdout.encoding
is utf-8, and you .decode(self.stdout.encoding,'replace')
A workaround is to always .decode() to ascii, and if that is what should
be done it should be documented at
https://docs.djangoproject.com/en/dev/howto/custom-management-commands/
/myapp/management/commands/unicode_test.py
{{{
# -*- coding: utf-8 -*-
from django.core.management.base import BaseCommand
class Command(BaseCommand):
args = '<none>'
help = 'Do something'
def handle(self, *args, **options):
output = u'Σάββα▒^▒ο'
self.stdout.write('Output: {}'.format(output))
}}}
{{{
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/__init__.py", line 399, in
execute_from_command_line
utility.execute()
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/home/tech/esldj/all-in-
one/go4/management/commands/unicode_test.py", line 10, in handle
self.stdout.write('Output: {}'.format(output))
File "/home/tech/.pyenv/versions/esldj-all-in-one-2.7.6/lib/python2.7
/site-packages/django/core/management/base.py", line 65, in write
if ending and not msg.endswith(ending):
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/21933#comment:2>
Django
Reporter: synotna | Owner: nobody
Type: Uncategorized | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by dhobbs):
Updated code as per the comment - this issue still exists.
--
Ticket URL: <https://code.djangoproject.com/ticket/21933#comment:3>
Django
Reporter: synotna | Owner: nobody
Type: Uncategorized | Status: closed
Component: Uncategorized | Version: 1.8
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by dhobbs):
* status: new => closed
* resolution: => invalid
Comment:
My bad - the issue does exist, but the example code here doesn't properly
demonstrate it.
--
Ticket URL: <https://code.djangoproject.com/ticket/21933#comment:4>
Django
Reporter: synotna | Owner: nobody
Type: Uncategorized | Status: closed
Component: Uncategorized | Version: 1.8
Severity: Normal | Resolution: invalid
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
The follow up ticket is #26731.
--
Ticket URL: <https://code.djangoproject.com/ticket/21933#comment:5>