[Django] #36705: Repeated string concatenation is slow on some python implementations, in particular PyPy

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Type:
| Cleanup/optimization
Status: new | Component: Utilities
Version: dev | Severity: Normal
Keywords: PyPy | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
There is a [https://pypy.org/posts/2023/01/string-concatenation-
quadratic.html known limitation] in PyPy that repeated string
concatenation is slow. The discussion there observes that CPython has an
optimization, but it's fragile and best not relied on.

Seokchan Yoon (thanks for the report!) raised this with the security team,
and we decided any hardening here could be done in public.

To close this ticket, we should:
- implement targeted optimizations in django.utils.numberformat.format()
to avoid repeated string concatenation
- audit for other similar cases
- Add a disclaimer to our
[https://docs.djangoproject.com/en/5.2/topics/performance/#id1
documentation] around PyPy, for example, wording like:

> That said, a lot of a web framework's work is done by concatenating
strings, and PyPy has an issue with that (see
https://pypy.org/posts/2023/01/string-concatenation-quadratic.html). This
may cause performance issues, depending on your use.
--
Ticket URL: <https://code.djangoproject.com/ticket/36705>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy

--------------------------------------+------------------------------------
Reporter: Jacob Walls | Owner: (none)

Type: Cleanup/optimization | Status: new
Component: Utilities | Version: dev

Severity: Normal | Resolution:
Keywords: PyPy | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Natalia Bidart):

* stage: Unreviewed => Accepted

Comment:

Thank you Jacob!
--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:1>

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy

-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Varun
Type: | Kasyap Pentamaraju
Cleanup/optimization | Status: assigned

Component: Utilities | Version: dev
Severity: Normal | Resolution:
Keywords: PyPy | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Varun Kasyap Pentamaraju):

* owner: (none) => Varun Kasyap Pentamaraju
* status: new => assigned

Comment:

I am willing to contribute
--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:2>

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Varun
Type: | Kasyap Pentamaraju
Cleanup/optimization | Status: assigned
Component: Utilities | Version: dev
Severity: Normal | Resolution:
Keywords: PyPy | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Varun Kasyap Pentamaraju):

* has_patch: 0 => 1

Comment:

Created PR https://github.com/django/django/pull/20045
--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:3>

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Varun
Type: | Kasyap Pentamaraju
Cleanup/optimization | Status: assigned
Component: Utilities | Version: dev
Severity: Normal | Resolution:
Keywords: PyPy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:4>

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Varun
Type: | Kasyap Pentamaraju
Cleanup/optimization | Status: assigned
Component: Utilities | Version: dev
Severity: Normal | Resolution:
Keywords: PyPy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Varun Kasyap Pentamaraju):

* needs_better_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:5>

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Varun
Type: | Kasyap Pentamaraju
Cleanup/optimization | Status: assigned
Component: Utilities | Version: dev
Severity: Normal | Resolution:
Keywords: PyPy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:6>

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Varun
Type: | Kasyap Pentamaraju
Cleanup/optimization | Status: assigned
Component: Utilities | Version: dev
Severity: Normal | Resolution:
Keywords: PyPy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Varun Kasyap Pentamaraju):

* needs_better_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:7>

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Varun
Type: | Kasyap Pentamaraju
Cleanup/optimization | Status: assigned
Component: Utilities | Version: dev
Severity: Normal | Resolution:
Keywords: PyPy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:8>

[Django]

#36705: Repeated string concatenation is slow on some python implementations, in
particular PyPy
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Varun
Type: | Kasyap Pentamaraju
Cleanup/optimization | Status: assigned
Component: Utilities | Version: dev
Severity: Normal | Resolution:

Keywords: PyPy | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Jacob Walls):

* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36705#comment:9>