[Django] #36737: Escape C1 control sequence in `escapejs`

[Django]

#36737: Escape C1 control sequence in `escapejs`
-------------------------------------+-------------------------------------
Reporter: Thibaut Decombe | Type:
| Cleanup/optimization
Status: new | Component: Template
| system
Version: 5.2 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
The current implementation of the
`escapejs`https://github.com/django/django/blob/5c60763561c67924eff1069e1516b60a59d068d5/django/utils/html.py#L79-L80
escapes only C0 control characters (unicode values ranging from 0 to 31)

However, there are other control characters in the 127-159 range, the C1
control characters.
See https://en.wikipedia.org/wiki/C0_and_C1_control_codes#C1_controls

Should we escape these too ?

The rust helper `char.is_control` https://doc.rust-
lang.org/src/core/char/methods.rs.html#952 consider both these ranges and
we were considering using it in django_rusty_templates

I'll be happy to provide a PR if it make sense
--
Ticket URL: <https://code.djangoproject.com/ticket/36737>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36737: Escape C1 control sequence in `escapejs`
-------------------------------------+-------------------------------------

Reporter: Thibaut Decombe | Owner: (none)
Type: | Status: new
Cleanup/optimization |

Component: Template system | Version: 5.2

Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Rudraksha Dwivedi):

While I am new to contributing to Django, it looks like a straightforward
job
the proposal does seem legit to me
escapejs already escapes C0 control characters, extending it to support C1
range would be better for consistency across control chars. in diff.
languages
if there is no backward compatibility issue
and the ticket gets accepted I'd love to chip in
--
Ticket URL: <https://code.djangoproject.com/ticket/36737#comment:1>

[Django]

#36737: Escape C1 control sequence in `escapejs`
-------------------------------------+-------------------------------------
Reporter: Thibaut Decombe | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Template system | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by farthestmage):

Hey I am also new to contributing the thought process it seems that a
simple loop can save the job
--
Ticket URL: <https://code.djangoproject.com/ticket/36737#comment:2>

[Django]

#36737: Escape C1 control sequence in `escapejs`
-------------------------------------+-------------------------------------
Reporter: Thibaut Decombe | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Template system | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by farthestmage):

Hey,
I was not assigned this task but majorly I believe I have solved the issue
and issued a pull request on github could pls review the code
thanks
--
Ticket URL: <https://code.djangoproject.com/ticket/36737#comment:3>

[Django]

#36737: Escape C1 control sequence in `escapejs`
-------------------------------------+-------------------------------------
Reporter: Thibaut Decombe | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Template system | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by farthestmage):

Hello,
A doubt were you able to produce or implement a way to test it out
--
Ticket URL: <https://code.djangoproject.com/ticket/36737#comment:4>

[Django]

#36737: Escape C1 control sequence in `escapejs`

---------------------------------+----------------------------------------
Reporter: Thibaut Decombe | Owner: farthestmage
Type: Bug | Status: assigned

Component: Template system | Version: 5.2
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 1 | Patch needs improvement: 1

Easy pickings: 1 | UI/UX: 0

---------------------------------+----------------------------------------
Changes (by Jacob Walls):

* needs_better_patch: 0 => 1
* needs_tests: 0 => 1
* owner: (none) => farthestmage
* stage: Unreviewed => Accepted
* status: new => assigned
* type: Cleanup/optimization => Bug

Comment:

Okay, I think this is right.

The [https://html.spec.whatwg.org/multipage/parsing.html HTML parsing
standard] describes the parsing error `control-character-in-input-stream`
like this:

> This error occurs if the input stream contains a control code point that
is not ASCII whitespace or U+0000 NULL. Such code points are parsed as-is
and usually, where parsing rules don't apply any additional restrictions,
make their way into the DOM.

[https://infra.spec.whatwg.org/#control control] is defined here:

> A control is a C0 control or a code point in the range U+007F DELETE to
U+009F APPLICATION PROGRAM COMMAND, inclusive.

The C1 control range is `U+0080 – U+009F`, so the additional characters
that need escaping are C1 control characters plus `U+007F DELETE` (which
is apparently sometimes grouped with C0 controls).

See also [https://www.w3.org/TR/2021/NOTE-html53-20210128/syntax.html
#preprocessing-the-input-stream HTML spec]

PR doesn't escape `U+007F DELETE` and needs tests.
--
Ticket URL: <https://code.djangoproject.com/ticket/36737#comment:5>

[Django]

#36737: Escape C1 control sequence in `escapejs`
---------------------------------+----------------------------------------
Reporter: Thibaut Decombe | Owner: farthestmage
Type: Bug | Status: assigned
Component: Template system | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
---------------------------------+----------------------------------------

Comment (by farthestmage):

Hey,
While creating Test cases I run runtests.py it works perfectly but while
review fails returns this error `AssertionError: '\\u009F' != '\x9f'`.I
have manipulated `\u009f` with taking it in a literal sense different
error arise while running `runtests.py` while the current commit creates
error only while reviewing not while executing `runtests.py`
help would be appreciated
thanks

PS: Just a new contributor learning
--
Ticket URL: <https://code.djangoproject.com/ticket/36737#comment:6>

[Django]

#36737: Escape C1 control sequence in `escapejs`

-------------------------------------+-------------------------------------

Reporter: Thibaut Decombe | Owner:
| farthestmage
Type: Bug | Status: assigned
Component: Template system | Version: 5.2
Severity: Normal | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 1 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Jacob Walls):

* needs_better_patch: 1 => 0
* needs_tests: 1 => 0
* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36737#comment:7>

[Django]

#36737: Escape C1 control sequence in `escapejs`
-------------------------------------+-------------------------------------
Reporter: Thibaut Decombe | Owner:
| farthestmage

Type: Bug | Status: closed

Component: Template system | Version: 5.2

Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls <jacobtylerwalls@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"07419875685997a30cd281396e0dc867e98aefe3" 07419875]:
{{{#!CommitTicketReference repository=""
revision="07419875685997a30cd281396e0dc867e98aefe3"
Fixed #36737 -- Escaped further control characters in escapejs.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36737#comment:8>