Re: [Django] #10449: HTML accents not escaped out when using forms

[Django]

#10449: HTML accents not escaped out when using forms
--------------------------------------+------------------------------------
Reporter: tipan | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.0
Severity: Normal | Resolution:
Keywords: accents, newforms | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------

Comment (by Guille López):

Is this bug unsolved yet? I would like to claim this to finally fix it.

--
Ticket URL: <https://code.djangoproject.com/ticket/10449#comment:16>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#10449: HTML accents not escaped out when using forms
--------------------------------------+------------------------------------
Reporter: tipan | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.0
Severity: Normal | Resolution:
Keywords: accents, newforms | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------

Changes (by Guille López):

* cc: Guille López (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/10449#comment:17>

[Django]

#10449: HTML accents not escaped out when using forms
--------------------------------------+------------------------------------
Reporter: tipan | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.0
Severity: Normal | Resolution:
Keywords: accents, newforms | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------

Comment (by Tim Graham):

Yes, the ticket is open which means the issue isn't resolved.

--
Ticket URL: <https://code.djangoproject.com/ticket/10449#comment:18>

[Django]

#10449: HTML accents not escaped out when using forms

-------------------------------------+-------------------------------------
Reporter: tipan | Owner: Abhishek
| Mane
Type: Bug | Status: assigned
Component: | Version: 1.0
Internationalization |

Severity: Normal | Resolution:
Keywords: accents, newforms | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Abhishek Mane):

* has_patch: 0 => 1
* needs_tests: 1 => 0
* owner: nobody => Abhishek Mane
* status: new => assigned

Comment:

I investigated this issue and found that modern Django's template-based
widget rendering no longer suffers from this double-escaping bug, provided
the translated strings are explicitly marked as safe. >
I have submitted a PR that adds a regression test to test_select.py to
prove mark_safe correctly prevents double-escaping in Select choices. This
fulfills the "Needs tests" requirement and proves the bug is resolved so
this ticket can finally be closed:
[https://github.com/django/django/pull/20871]
--
Ticket URL: <https://code.djangoproject.com/ticket/10449#comment:19>

[Django]

#10449: HTML accents not escaped out when using forms
-------------------------------------+-------------------------------------
Reporter: tipan | Owner: Abhishek
| Mane
Type: Bug | Status: assigned
Component: | Version: 1.0
Internationalization |
Severity: Normal | Resolution:
Keywords: accents, newforms | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* has_patch: 1 => 0
* needs_docs: 1 => 0

Comment:

PR from comment#19 just documented (via a test case) calling `mark_safe()`
on hard-coded values sporting HTML entities, whereas I took the point to
be that translation strings ''might or might not'' contain HTML entities
after resolving them to local values.
--
Ticket URL: <https://code.djangoproject.com/ticket/10449#comment:20>

[Django]

#10449: HTML accents not escaped out when using forms
-------------------------------------+-------------------------------------
Reporter: tipan | Owner: Abhishek
| Mane
Type: Bug | Status: assigned
Component: | Version: 1.0
Internationalization |
Severity: Normal | Resolution:
Keywords: accents, newforms | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Abhishek Mane):

Hi Jacob, thanks for the review and for closing the PR. Your point makes
complete sense,I was definitely just memorializing a gotcha rather than
fixing the root issue. You are completely right that forcing developers to
sprinkle mark_safe() everywhere is terrible DX and opens up massive XSS
risks if a .po file contains malicious HTML tags.

I want to take a step back and figure out the right architectural approach
before writing a new patch. As I see it, we have three potential paths
forward:

1. Selective Entity Decoding: Modify the translation wrapper to only
unescape standard safe HTML entities (like &amp; or &copy;) from the
localized string, while keeping actual HTML tags (like <script>) strictly
escaped.

2. A Dedicated Template Filter: Introduce a new filter (e.g.,
|safe_entities) that developers can use in templates when they suspect a
translator might have injected entities. This avoids turning off full
auto-escaping via mark_safe().

3. WONTFIX / Documentation Only: Is this fundamentally a translator
education issue? If a translator uses a literal & instead of &amp; in the
.po file, Django's auto-escaper handles it perfectly in the template.
Should we just officially document that translators must use literal
characters rather than HTML entities unless the developer specifically
intended the string to contain HTML?

I'd love to get the team's thoughts on which of these directions (or
another one I missed) is the right path forward so I can put together a
proper PR.
--
Ticket URL: <https://code.djangoproject.com/ticket/10449#comment:21>