[Django] #30184: Django 2.2b1 throws error on Windows in default code page
Django
-----------------------------------------+---------------------------
Reporter: Dan Davis | Owner: Dan Davis
Type: Uncategorized | Status: assigned
Component: Core (Other) | Version: 2.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-----------------------------------------+---------------------------
By default, Windows terminals start with code page 437, which is roughly
equivalent to ASCII. While working on #30178, I discovered that this
causes the following traceback:
```
(mplusmon) λ manage.py runserver
Watchman unavailable: pywatchman not installed.
Watching for file changes with StatReloader
Exception in thread Thread-1:
Traceback (most recent call last):
File "c:\tools\python\3.5\Lib\threading.py", line 914, in
_bootstrap_inner
self.run()
File "c:\tools\python\3.5\Lib\threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\davisda4\PythonEnvs\mplusmon\lib\site-
packages\django\utils\autoreload.py", line 54, in wrapper
fn(*args, **kwargs)
File "C:\Users\davisda4\PythonEnvs\mplusmon\lib\site-
packages\django\core\management\commands\runserver.py", line 116, in
inner_run
self.stdout.write("Performing system checks\u2026\n\n")
File "C:\Users\davisda4\PythonEnvs\mplusmon\lib\site-
packages\django\core\management\base.py", line 145, in write
self._out.write(style_func(msg))
File "C:\Users\davisda4\PythonEnvs\mplusmon\lib\encodings\cp437.py",
line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2026' in
position 24: character maps to <undefined>
```
The problem can be resolved by changing the code page to UTF-8, as
follows:
```
chcp 65001
```
This appears to be a new issue relative to Django 2.1.7, because the
problem did not occur there. I'll track down wherever someone entered
the horizontal elipsis character rather than "..." into the code, because
this ought to be easy enough to fix.
--
Ticket URL: <https://code.djangoproject.com/ticket/30184>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Dan Davis | Owner: Dan Davis
Type: Uncategorized | Status: assigned
Component: Core (Other) | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Description changed by Dan Davis:
Old description:
New description:
A workaround is to change the code page to UTF-8, as follows:
{{{
chcp 65001
}}}
This appears to be a new issue relative to Django 2.1.7, because the
problem did not occur there. I'll track down wherever someone entered
the horizontal elipsis character rather than "..." into the code, because
this ought to be easy enough to fix. Whomever did this was probably not
even aware of it and it has worked well enough since almost all Django
developers will use MacOS or Linux.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:1>
Django
------------------------------+-------------------------------------
Reporter: Dan Davis | Owner: Dan Davis
Component: Core (Other) | Version: 2.2
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
------------------------------+-------------------------------------
Changes (by Tim Graham):
* type: Uncategorized => Bug
* easy: 1 => 0
* stage: Unreviewed => Accepted
Comment:
It was changed in #29654.
I also saw a report of the crash on [https://groups.google.com/d/topic
/django-users/eeJgrSa6eoU/discussion django-users].
Since there's not much benefit to the ellipses characters compared to
three dots, I think it's fine to revert the problematic parts.
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:2>
Django
Reporter: Dan Davis | Owner: Dan Davis
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Changes (by Dan Davis):
* component: Core (Other) => Core (Management commands)
* easy: 0 => 1
Comment:
Ellipsis is in runserver command.
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:3>
Django
Reporter: Dan Davis | Owner: Dan Davis
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Comment (by Dan Davis):
Pull request - https://github.com/django/django/pull/10989
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:4>
Django
Reporter: Dan Davis | Owner: Dan Davis
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):
* needs_better_patch: 0 => 1
* has_patch: 0 => 1
* easy: 1 => 0
Comment:
Please look at the linked ticket and uncheck "Patch needs improvement"
after reverting all strings that might be output to the shell.
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:5>
Django
-------------------------------------+-------------------------------------
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Dan Davis):
I will do that hopefully Thursday. Tomorrow is cub scout den meeting and
I must prepare ;)
I maybe can write a unit test that captures output from management
commands and tests them, but that maybe a little much.
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:6>
Django
-------------------------------------+-------------------------------------
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
Comment:
Fun with ast module -
https://gist.github.com/danizen/e8c53a36af6174f88194c5b1bd3e7a3f
That would probably be overmuch as a unit test. Still, all core
management commands and contrib management commands are now cp437 safe.
Let me know if I rebased properly:
{{{
git rebase -i HEAD~2
git pull upstream master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:7>
Django
-------------------------------------+-------------------------------------
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Dan Davis):
Replying to [comment:2 Tim Graham]:
> It was changed in #29654.
Then I shall also check database creations. it is of course true that any
"engineer" worth their salt should be unicode savvy, but DjangoCon 2016
convinced me that Django's tutorial should work just fine for a 15 year
old programmer using their Mom's older Windows 7 Professional laptop.
Fortunate for me that matches up with the desktops my users are
constrained to use at work :[
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:8>
Django
-------------------------------------+-------------------------------------
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* cc: jon.dufresne@… (added)
Comment:
Is there a larger problem that ''any'' Unicode output will fail on
Windows? If so, is this just avoiding a single case rather than solving a
deeper problem? Isn't it reasonable to expect some other Unicode output
down the road?
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:9>
Django
-------------------------------------+-------------------------------------
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham):
Perhaps we could try to make Django not crash and have it instead output
some instructions about how to correct a shell configuration to support
Unicode. But that would be a separate low priority enhancement, and I
still wouldn't put non-ASCII characters like ellipses in output strings
when a suitable alternative exists. I haven't seen any reports of this
sort of crash before.
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:10>
Django
-------------------------------------+-------------------------------------
Component: Core (Management | Version: 2.2
commands) |
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"2bd8df243ac6fc35e58c9fe90b20c9e42519a5ac" 2bd8df24]:
{{{
#!CommitTicketReference repository=""
revision="2bd8df243ac6fc35e58c9fe90b20c9e42519a5ac"
Fixed #30184 -- Removed ellipsis characters from shell output strings.
Partially reverted 50b8493581fea3d7137dd8db33bac7008868d23a (refs #29654)
to avoid a crash when the user shell doesn't support non-ASCII characters.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:11>
Django
-------------------------------------+-------------------------------------
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"5e94c817ee26a13e141a11a2f429f9ee2cc37ab0" 5e94c817]:
{{{
#!CommitTicketReference repository=""
revision="5e94c817ee26a13e141a11a2f429f9ee2cc37ab0"
[2.2.x] Fixed #30184 -- Removed ellipsis characters from shell output
strings.
Partially reverted 50b8493581fea3d7137dd8db33bac7008868d23a (refs #29654)
to avoid a crash when the user shell doesn't support non-ASCII characters.
Backport of 2bd8df243ac6fc35e58c9fe90b20c9e42519a5ac from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:12>
Django
-------------------------------------+-------------------------------------
Component: Core (Management | Version: 2.2
commands) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Claude Paroz):
We are in 2019, and there still are terminals not supporting Unicode, sad
:-(
--
Ticket URL: <https://code.djangoproject.com/ticket/30184#comment:13>