[Django] #22880: UnicodeDecode error when running `makemessages` (python 3.4.1)
Django
--------------------------------------------+------------------------
Reporter: valberg | Owner: nobody
Type: Uncategorized | Status: new
Component: Core (Management commands) | Version: 1.7-beta-2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------------+------------------------
Made this model for testing:
{{{#!python
from django.db import models
from django.utils.translation import ugettext_lazy as _
class Foo(models.Model):
bar = models.CharField(
verbose_name=_('bar'),
max_length=255
)
}}}
`django-admin makemessages -l da` creates the following `django.po` file:
{{{#!po
$ cat locale/da/LC_MESSAGES/django.po
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2014-06-22 19:49+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <L...@li.org>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
#: models.py:7
msgid "bar"
msgstr ""
}}}
Changing the last line (doing the translation of "bar") to this:
{{{#!po
msgstr "æøå"
}}}
Results in this error when running `django-admin makemessages -l da`
again:
{{{
processing locale da
Traceback (most recent call last):
File "./../manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/valberg/.virtualenvs/django17/lib/python3.4/site-
packages/django/core/management/__init__.py", line 385, in
execute_from_command_line
utility.execute()
File "/Users/valberg/.virtualenvs/django17/lib/python3.4/site-
packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/valberg/.virtualenvs/django17/lib/python3.4/site-
packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/valberg/.virtualenvs/django17/lib/python3.4/site-
packages/django/core/management/base.py", line 337, in execute
output = self.handle(*args, **options)
File "/Users/valberg/.virtualenvs/django17/lib/python3.4/site-
packages/django/core/management/base.py", line 532, in handle
return self.handle_noargs(**options)
File "/Users/valberg/.virtualenvs/django17/lib/python3.4/site-
packages/django/core/management/commands/makemessages.py", line 295, in
handle_noargs
self.write_po_file(potfile, locale)
File "/Users/valberg/.virtualenvs/django17/lib/python3.4/site-
packages/django/core/management/commands/makemessages.py", line 397, in
write_po_file
msgs, errors, status = popen_wrapper(args)
File "/Users/valberg/.virtualenvs/django17/lib/python3.4/site-
packages/django/core/management/utils.py", line 25, in popen_wrapper
output, errors = p.communicate()
File "/Users/valberg/.pyenv/versions/3.4.1/lib/python3.4/subprocess.py",
line 959, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/Users/valberg/.pyenv/versions/3.4.1/lib/python3.4/subprocess.py",
line 1658, in _communicate
self.stdout.encoding)
File "/Users/valberg/.pyenv/versions/3.4.1/lib/python3.4/subprocess.py",
line 887, in _translate_newlines
data = data.decode(encoding)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 663:
ordinal not in range(128)
}}}
I'm not sure whether this is a django error or a python one.
--
Ticket URL: <https://code.djangoproject.com/ticket/22880>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: valberg | Owner: nobody
Type: Uncategorized | Status: new
commands) | 1.7-beta-2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
Has patch: 0 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by claudep):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
Did you try with other versions of Django?
--
Ticket URL: <https://code.djangoproject.com/ticket/22880#comment:1>
Django
Reporter: valberg | Owner: nobody
Type: Uncategorized | Status: new
commands) | 1.7-beta-2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
Has patch: 0 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by valberg):
Just did, and it also happens with 1.6.5 and 1.5.8.
--
Ticket URL: <https://code.djangoproject.com/ticket/22880#comment:2>
Django
Reporter: valberg | Owner: nobody
Type: Uncategorized | Status: new
commands) | 1.7-beta-2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
Has patch: 0 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by valberg):
It looks as if my locale was wrong somehow:
{{{
$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=
}}}
Setting `LANG` to `"da_DK.UTF-8"` does the trick as far as I can tell.
{{{
$ export LANG="da_DK.UTF-8"
$ locale
LANG="da_DK.UTF-8"
LC_COLLATE="da_DK.UTF-8"
LC_CTYPE="da_DK.UTF-8"
LC_MESSAGES="da_DK.UTF-8"
LC_MONETARY="da_DK.UTF-8"
LC_NUMERIC="da_DK.UTF-8"
LC_TIME="da_DK.UTF-8"
LC_ALL="da_DK.UTF-8"
$ django-admin makemessages --traceback -l da
processing language da
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22880#comment:3>
Django
Reporter: valberg | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version:
Component: Documentation | 1.7-beta-2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
* component: Core (Management commands) => Documentation
* has_patch: 0 => 1
* type: Uncategorized => Cleanup/optimization
Comment:
PR: https://github.com/django/django/pull/2841
--
Ticket URL: <https://code.djangoproject.com/ticket/22880#comment:4>
Django
Reporter: valberg | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version:
Component: Documentation | 1.7-beta-2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by claudep):
After the night :-), here's how I would have written it:
https://github.com/django/django/pull/2842
--
Ticket URL: <https://code.djangoproject.com/ticket/22880#comment:5>
Django
Reporter: valberg | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version:
Component: Documentation | 1.7-beta-2
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timo):
* stage: Unreviewed => Accepted
Comment:
It's not really a Django specific issue, but I guess it wouldn't hurt to
mention as we already have a topic guide on unicode.
--
Ticket URL: <https://code.djangoproject.com/ticket/22880#comment:6>
Django
Reporter: valberg | Owner: nobody
Type: | Status: closed
Cleanup/optimization | Version:
Component: Documentation | 1.7-beta-2
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Claude Paroz <claude@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"460ec09d2ed26f5862aabeaa480bcb713d5f331f"]:
{{{
#!CommitTicketReference repository=""
revision="460ec09d2ed26f5862aabeaa480bcb713d5f331f"
Fixed #22880 -- Added FAQ entry about UnicodeDecodeError
Thanks Víðir Valberg Guðmundsson for the report and
Tim Graham for the review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22880#comment:7>
Django
Reporter: valberg | Owner: nobody
Type: | Status: closed
Cleanup/optimization | Version:
Component: Documentation | 1.7-beta-2
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Claude Paroz <claude@…>):
In [changeset:"198ce3073c763ab17cb5e8d012d64d8ea756282d"]:
{{{
#!CommitTicketReference repository=""
revision="198ce3073c763ab17cb5e8d012d64d8ea756282d"
[1.7.x] Fixed #22880 -- Added FAQ entry about UnicodeDecodeError
Thanks Víðir Valberg Guðmundsson for the report and
Tim Graham for the review.
Backport of 460ec09d2e from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22880#comment:8>