[Django] #36867: Optimize template compilation with intelligent caching for repeated renders

[Django]

#36867: Optimize template compilation with intelligent caching for repeated renders
-------------------------------------+-------------------------------------
Reporter: TheAshutoshMishra | Type: New
| feature
Status: new | Component: Template
| system
Version: 6.0 | Severity: Normal
Keywords: templates, | Triage Stage:
performance, caching, | Unreviewed
optimization |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Currently, Django's template system performs expensive compilation
operations (tokenization and filter expression parsing) on every template
render, even when rendering the same template repeatedly with different
contexts. This causes unnecessary CPU overhead in production.

=== Problem ===

The template compilation process involves:
1. **Tokenization**: Regex splitting of template strings via
`tag_re.split()`
2. **Filter expression compilation**: Parsing filter syntax like `{{
var|filter:"arg" }}`

Both operations are repeated identically for every render, even though the
results are deterministic based on the template source.

=== Proposed Solution ===

Add bounded, memory-safe caching for:
1. **Tokenized template strings** (max 1000 entries, hash-based keys)
2. **Compiled FilterExpression objects** (max 500 entries)

=== Implementation Details ===

- Hash-based cache keys for O(1) lookups
- Smart eviction: removes 20-25% of oldest entries when limit reached
- Skips caching for stateful verbatim templates
- Graceful fallbacks for edge cases
- New `clear_template_caches()` utility for production cache management

=== Performance Impact (Measured) ===

- **13-47% faster** template rendering on repeated renders
- **1.15M tokenizations/second** throughput with caching
- Zero configuration required
- Fully backward compatible

=== PR ===

Implementation available in PR #20540:
https://github.com/django/django/pull/20540

Looking for feedback on this approach and guidance on adding comprehensive
test coverage.
--
Ticket URL: <https://code.djangoproject.com/ticket/36867>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36867: Optimize template compilation with intelligent caching for repeated renders
-------------------------------------+-------------------------------------

Reporter: TheAshutoshMishra | Owner: (none)
Type: New feature | Status: closed

Component: Template system | Version: 6.0

Severity: Normal | Resolution: duplicate

Keywords: templates, | Triage Stage:
performance, caching, | Unreviewed
optimization |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* resolution: => duplicate
* status: new => closed

Comment:

Identical duplicate of #36866. Please avoid use of LLM to fully write
tickets and PRs. If an LLM is used as an **assistance**, please carefully
review its outcome and also replace the overly verbose output with
concise, correct, and precise information.
--
Ticket URL: <https://code.djangoproject.com/ticket/36867#comment:1>