[Django] #29246: compilemessages management-command fails with UnicodeDecodeError
Django
-------------------------------------+-------------------------------------
Reporter: Tarun | Owner: nobody
Gaba |
Type: Bug | Status: new
Component: | Version: 1.9
Uncategorized | Keywords: compilemessages
Severity: Normal | UnicodeDecodeError
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-------------------------------------+-------------------------------------
When running compilemessages command on a django project which contains
Unicode characters in filenames (for example in static files), it fails
with a UnicodeError:
{{{
Traceback (most recent call last):
File "manage.py", line 62, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-
packages/django/core/management/__init__.py", line 364, in
execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-
packages/django/core/management/__init__.py", line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-
packages/django/core/management/base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python2.7/dist-
packages/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-
packages/django/core/management/commands/compilemessages.py", line 70, in
handle
for dirpath, dirnames, filenames in os.walk('.', topdown=True):
File "/usr/lib/python2.7/os.py", line 294, in walk
for x in walk(new_path, topdown, onerror, followlinks):
File "/usr/lib/python2.7/os.py", line 294, in walk
for x in walk(new_path, topdown, onerror, followlinks):
File "/usr/lib/python2.7/os.py", line 284, in walk
if isdir(join(top, name)):
File "/usr/lib/python2.7/posixpath.py", line 80, in join
path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 23:
ordinal not in range(128)
}}}
The 'walk' functionality in `compilemessages` was introduced in django
`1.9`:
(https://github.com/django/django/blob/master/django/core/management/commands/compilemessages.py#L67)
--
Ticket URL: <https://code.djangoproject.com/ticket/29246>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: | Version: 1.9
Internationalization |
Severity: Normal | Resolution: invalid
Keywords: compilemessages | Triage Stage:
UnicodeDecodeError | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* component: Uncategorized => Internationalization
* resolution: => invalid
Comment:
Django 1.9 is no longer supported. Please reopen if you can reproduce with
Django's master branch, which supports only Python 3.
--
Ticket URL: <https://code.djangoproject.com/ticket/29246#comment:1>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: | Version: 1.11
Internationalization |
Severity: Normal | Resolution:
Keywords: compilemessages | Triage Stage:
UnicodeDecodeError | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* cc: Daniel Browne (added)
* version: 1.9 => 1.11
* resolution: invalid =>
Comment:
I can reproduce this issue on Django 1.11 which is still supported. Can
the '.' passed to os.walk not become b'.' in order to bypass the
unicode_literals import? I think this would solve the problem.
(https://github.com/django/django/blob/stable/1.11.x/django/core/management/commands/compilemessages.py#L70)
--
Ticket URL: <https://code.djangoproject.com/ticket/29246#comment:2>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: | Version: 1.11
Internationalization |
Severity: Normal | Resolution:
Keywords: compilemessages | Triage Stage:
UnicodeDecodeError | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham):
Per our [https://docs.djangoproject.com/en/dev/internals/release-process
/#supported-versions supported versions policy], 1.11 is only receiving
data loss and security fixes. Can the issue be reproduce on Django's
master branch (which supports Python 3 only).
--
Ticket URL: <https://code.djangoproject.com/ticket/29246#comment:3>
Django
-------------------------------------+-------------------------------------
| Qin
Type: Bug | Status: assigned
Component: | Version: 1.11
Internationalization |
Severity: Normal | Resolution:
Keywords: compilemessages | Triage Stage:
UnicodeDecodeError | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Liuyang Qin
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/29246#comment:4>
Django
-------------------------------------+-------------------------------------
| Qin
Component: | Version: 1.11
Internationalization |
Severity: Normal | Resolution: needsinfo
Keywords: compilemessages | Triage Stage:
UnicodeDecodeError | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => needsinfo
--
Ticket URL: <https://code.djangoproject.com/ticket/29246#comment:5>
Django
-------------------------------------+-------------------------------------
| Qin
Type: Bug | Status: closed
Component: | Version: 1.11
Internationalization |
Severity: Normal | Resolution: needsinfo
Keywords: compilemessages | Triage Stage:
UnicodeDecodeError | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by digitalfox):
Got the same bug on django 1.11. One simple way to workaround this is to
cd to your locale/ directory before calling compilemessages. Btw that also
speed up things, avoiding to search locale folders everywhere.
--
Ticket URL: <https://code.djangoproject.com/ticket/29246#comment:6>