[Django] #35675: Reduce impact of parsing crafted templates with repeat tags

[Django]

#35675: Reduce impact of parsing crafted templates with repeat tags
-----------------------------+-------------------------------------------
Reporter: Jake Howard | Type: Bug
Status: new | Component: Template system
Version: | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------+-------------------------------------------
The template system uses a regex to extract template tags from text. Given
certain inputs, this can take an excessive amount of time:

{{{#!python
In [2]: %timeit Template("{%" * 2000)
34.7 ms ± 153 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [3]: %timeit Template("{%" * 10000)
877 ms ± 1.49 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [4]: %timeit Template("{%" * 20000)
3.49 s ± 47 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [5]: %timeit Template("{%")
11.5 µs ± 55.3 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops
each)
}}}

The cause is excessive backtracking in the
[https://github.com/django/django/blob/b99c608ea10cabc97a6b251cdb6e81ef2a83bdcf/django/template/base.py#L89C23-L89C48
pattern used]. Since the template system is so versatile and performance-
critical, fixing the issue appears non-trivial.

Note: This bug was raised with the Security Team prior to opening, however
was not deemed a security vulnerability since parsing untrusted (or semi-
trusted) templates is
[https://docs.djangoproject.com/en/5.0/topics/templates/ explicitly warned
against].
--
Ticket URL: <https://code.djangoproject.com/ticket/35675>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35675: Reduce impact of parsing crafted templates with repeat tags

--------------------------------------+------------------------------------
Reporter: Jake Howard | Owner: (none)
Type: Cleanup/optimization | Status: new
Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Natalia Bidart):

* stage: Unreviewed => Accepted
* type: Bug => Cleanup/optimization
* version: => dev

Comment:

Thank you Jake for taking the time to create this report. Accepting
following the conversation within the Security Team.
--
Ticket URL: <https://code.djangoproject.com/ticket/35675#comment:1>

[Django]

#35675: Reduce impact of parsing crafted templates with repeat tags
--------------------------------------+------------------------------------
Reporter: Jake Howard | Owner: (none)
Type: Cleanup/optimization | Status: new
Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------

Comment (by Aditya Chaudhary):

Can't we use precompiling a regex pattern to enhance performance by
avoiding repeated compilation overhead for frequent matches?
--
Ticket URL: <https://code.djangoproject.com/ticket/35675#comment:2>

[Django]

#35675: Reduce impact of parsing crafted templates with repeat tags
--------------------------------------+------------------------------------
Reporter: Jake Howard | Owner: (none)
Type: Cleanup/optimization | Status: new
Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------

Comment (by Jake Howard):

> Can't we use precompiling a regex pattern

No, this regex is already compiled before execution (see the linked code
above). Repeat compiling isn't the performance hit here, it's the
backtracking in the pattern itself.
--
Ticket URL: <https://code.djangoproject.com/ticket/35675#comment:3>

[Django]

#35675: Reduce impact of parsing crafted templates with repeat tags

-------------------------------------+-------------------------------------
Reporter: Jake Howard | Owner: Hailey
Type: | Johnson
Cleanup/optimization | Status: assigned

Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Hailey Johnson):

* owner: (none) => Hailey Johnson
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35675#comment:4>

[Django]

#35675: Reduce impact of parsing crafted templates with repeat tags
-------------------------------------+-------------------------------------
Reporter: Jake Howard | Owner: Hailey
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Hailey Johnson):

* has_patch: 0 => 1

Comment:

See PR #19360[https://github.com/django/django/pull/19360]
--
Ticket URL: <https://code.djangoproject.com/ticket/35675#comment:5>

[Django]

#35675: Reduce impact of parsing crafted templates with repeat tags
-------------------------------------+-------------------------------------
Reporter: Jake Howard | Owner: Hailey
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 1 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* needs_tests: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/35675#comment:6>