[Django] #27146: template shows empty string instead of actual content when it contains characters like \x93 and \x94

[Django]

#27146: template shows empty string instead of actual content when it contains
characters like \x93 and \x94
---------------------------------+--------------------
Reporter: bricas | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 1.8
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------
I believe \x93 and \x94 are quote characters from CP-1252.

I can use `{{string|pprint}}` and see the data in variable, i can use
`{{string|length}}` and gets its length, but i cannot just print it via
`{{string}}` -- no error is thrown, but the output is empty.

This seems like a bug to me, but I can convinced otherwise. :)

--
Ticket URL: <https://code.djangoproject.com/ticket/27146>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#27146: template shows empty string instead of actual content when it contains
characters like \x93 and \x94

---------------------------------+--------------------------------------

Reporter: bricas | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 1.8

Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

---------------------------------+--------------------------------------
Changes (by timgraham):

* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0

Comment:

It would be quite nice if you could do some investigation yourself and
tell us where the bug is. Or even write a test for Django's test suite
(django/tests/template_tests) so we at least have an easy way to reproduce
it. In general, you should ask "is it a bug?" questions on our
[wiki:TicketClosingReasons/UseSupportChannels our support channels] rather
than creating a Trac ticket.

--
Ticket URL: <https://code.djangoproject.com/ticket/27146#comment:1>

[Django]

#27146: template shows empty string instead of actual content when it contains
characters like \x93 and \x94

---------------------------------+--------------------------------------

Reporter: bricas | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 1.8

Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

---------------------------------+--------------------------------------

Comment (by bricas):

I apologies for the lack of details. Python & Django are not things I use
on a daily basis.

I originally enquired about my issue in #django and didn't get much more
than a "that's strange" as a response.

I will spend a little time and try to write up a test.

--
Ticket URL: <https://code.djangoproject.com/ticket/27146#comment:2>

[Django]

#27146: template shows empty string instead of actual content when it contains
characters like \x93 and \x94

---------------------------------+--------------------------------------
Reporter: bricas | Owner: nobody
Type: Bug | Status: closed

Component: Template system | Version: 1.8

Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

---------------------------------+--------------------------------------
Changes (by timgraham):

* status: new => closed
* resolution: => needsinfo

Comment:

Thanks, I attached the quick test I wrote up to try to reproduce but it
passes on Python 2 and 3, so there's probably some missing information
about the exact steps to reproduce. Please reopen or change the resolution
based on your investigation.

--
Ticket URL: <https://code.djangoproject.com/ticket/27146#comment:3>

[Django]

#27146: template shows empty string instead of actual content when it contains
characters like \x93 and \x94

---------------------------------+--------------------------------------
Reporter: bricas | Owner: nobody
Type: Bug | Status: closed

Component: Template system | Version: 1.8

Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

---------------------------------+--------------------------------------
Changes (by timgraham):

* Attachment "27146-test.diff" added.

[Django]

#27146: template shows empty string instead of actual content when it contains
characters like \x93 and \x94

---------------------------------+--------------------------------------
Reporter: bricas | Owner: nobody
Type: Bug | Status: closed

Component: Template system | Version: 1.8

Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

---------------------------------+--------------------------------------

Comment (by bricas):

Thanks for the update -- I also saw the same result when i tried to add a
test. I suspect there are other encoding-related issues happening.

To elaborate on the *actual* scenario we're dealing with: We have a small
Django app that parses `git diff` output and shows what those changes are.
The files being diff'ed could come from a variety of OSes in who-knows-
what encoding (including CP-1252). I have a feeling that properly reading
in the diff output would solve the issue.

--
Ticket URL: <https://code.djangoproject.com/ticket/27146#comment:4>