[Django] #37214: Performance regression with end to end migrations on v5.x

[Django]

#37214: Performance regression with end to end migrations on v5.x
--------------------------+------------------------------------------------
Reporter: wyardley | Type: Cleanup/optimization
Status: new | Component: Migrations
Version: 5.2 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------+------------------------------------------------
We've recently updated from Django 4.2 to 5.2. We have a nightly job that
runs all our migrations end to end.
Even after squashing _all_ our migrations (basically declaring migration
bankruptcy fully) recently, there are ~ 800 migrations across ~ 95 apps.

The nightly run doubled in time from 45m to 90m just from this update.
Profiling a from-scratch migration shows about 93% of the time in
ProjectState rendering and almost none in cursor.execute.

Some quick Claude analysis (so the usual caveats apply) suggests this
could relate to changes with caching of swappable-settings lookups, and
_possibly_ also how field choices get normalized?

I know there have been some (long) past discussions mentioning that Django
isn't trying to optimize this path too much. But to me, it does seem like
a pretty significant regression for this to take twice as long.

Happy to provide other details if it's helpful; unfortunately, providing a
full clean reproduction may be difficult, since this is happening in a
fairly large internal codebase.

Possibly related: #34687 and #28033
--
Ticket URL: <https://code.djangoproject.com/ticket/37214>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#37214: Performance regression with end to end migrations on v5.x

-------------------------------------+-------------------------------------
Reporter: William Yardley | Owner: (none)
Type: | Status: closed
Cleanup/optimization |
Component: Migrations | Version: 5.2
Severity: Normal | Resolution: needsinfo

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Jacob Walls):

* resolution: => needsinfo
* status: new => closed

Comment:

Thanks for the report. Some additional information will be needed to move
this forward, I'm afraid. If you have a suspicion choices are involved,
then scripting out a test project with a dozen migrations that only tinker
with choices, and measuring if it's slower on 5.2 would help.
--
Ticket URL: <https://code.djangoproject.com/ticket/37214#comment:1>