[Django] #24670: Django migrations are not scaling well with number of operations in migration
Django
migration
--------------------------------------+------------------------------------
Reporter: kurevin | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Migrations | Version: 1.8
Severity: Normal | Keywords: migrations
| optimization
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
In our project there are around 200 models, most of them are in a single
app.
I've been testing potencial performance improvements of migration runner
in django 1.8.
When running migration that adds 17 fields this happens:
Rendering model states... DONE (23.982s)
Applying xxx... OK (28.930s)
50 seconds to add just 17 fields to a completely empty DB.
Similar results shows current django master.
In comparison, migration that adds a single field:
Rendering model states... DONE (2.994s)
Applying xxx2... OK (1.540s)
Profiling of 17 fields migration:
Rendering model states... DONE (27.341s)
Applying xxx... OK (33.287s)
{{{
26957269 function calls (26828616 primitive calls) in 61.414
seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.007 0.007 61.414 61.414
django/core/management/commands/migrate.py:55(handle)
1 0.000 0.000 60.634 60.634
django/db/migrations/executor.py:65(migrate)
525 0.003 0.000 51.490 0.098
django/db/migrations/operations/fields.py:43(state_forwards)
582 0.030 0.000 51.487 0.088
django/db/migrations/state.py:88(reload_model)
54 0.016 0.000 39.538 0.732
django/db/migrations/state.py:240(render_multiple)
6174 0.251 0.000 39.521 0.006
django/db/migrations/state.py:507(render)
1 0.005 0.005 33.287 33.287
django/db/migrations/executor.py:132(apply_migration)
1 0.055 0.055 33.260 33.260
django/db/migrations/migration.py:86(apply)
15525 1.869 0.000 25.929 0.002
django/apps/registry.py:334(clear_cache)
40 0.001 0.000 25.852 0.646
django/db/migrations/migration.py:72(mutate_state)
7889/6140 0.648 0.000 23.327 0.004
django/db/models/base.py:72(__new__)
137224 0.725 0.000 22.687 0.000
django/db/migrations/state.py:476(construct_fields)
2049773 19.569 0.000 19.569 0.000
django/db/models/options.py:711(_expire_cache)
7889 0.040 0.000 13.726 0.002
django/db/migrations/state.py:273(register_model)
7616 0.028 0.000 12.333 0.002
django/db/migrations/state.py:281(unregister_model)
25408 0.135 0.000 10.822 0.000
django/db/models/fields/related.py:1867(deconstruct)
25408 0.250 0.000 10.687 0.000
django/db/models/fields/related.py:1551(deconstruct)
28344 5.208 0.000 10.398 0.000
django/db/models/fields/related.py:309(swappable_setting)
43915 0.147 0.000 9.827 0.000
django/utils/lru_cache.py:94(wrapper)
21213 1.501 0.000 9.416 0.000
django/apps/registry.py:153(get_models)
118321/106078 0.231 0.000 9.062 0.000
django/db/models/base.py:321(add_to_class)
3985834 3.512 0.000 7.534 0.000
django/apps/config.py:164(get_models)
19 0.001 0.000 7.105 0.374
django/db/migrations/state.py:142(clone)
19 0.007 0.000 6.920 0.364
django/db/migrations/state.py:145(<dictcomp>)
3572 0.056 0.000 6.913 0.002
django/db/migrations/state.py:496(clone)
127520 4.815 0.000 5.706 0.000
django/db/models/fields/__init__.py:358(deconstruct)
1891 0.024 0.000 5.029 0.003
django/db/models/fields/related.py:2518(contribute_to_class)
1749 0.092 0.000 4.856 0.003
django/db/models/fields/related.py:2048(create_many_to_many_intermediary
_model)
741163 2.485 0.000 3.776 0.000
/usr/lib/python2.7/collections.py:104(values)
}}}
We also have a initial migration that adds ~400 FK fields and it takes up
to 10 minutes to execute.
--
Ticket URL: <https://code.djangoproject.com/ticket/24670>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
migration
Reporter: kurevin | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage:
optimization | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by knbk):
* cc: marten.knbk@… (added)
* needs_better_patch: => 0
* version: 1.8 => master
* needs_tests: => 0
* needs_docs: => 0
Comment:
Could you elaborate on what potential performance improvements you have
been testing? I've been working on some patches myself (see #24397,
#24590, #24591), but I'd be interested to know if there are any potential
optimizations that I've missed.
Thanks a lot for taking the time and effort to look into this, it is much
appreciated.
--
Ticket URL: <https://code.djangoproject.com/ticket/24670#comment:1>
Django
migration
Reporter: kurevin | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage:
optimization | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by kurevin):
* cc: kurevin (added)
Comment:
knbk, thanks for the useful links!
I was actually looking for performance improvements over Django 1.7.7,
which we are currently using. Our 1.7.7 has few major optimizations
backported from early 1.8, but it still slow, we might try to backport
some of mentioned tickets.
Just a guess if you are looking for more improvements - it seems that
runner does not cache inter-operation states, given that rendering time
and migration running time match quite closely, and there is a lot of
calls to state_forwards of AddField operation, 30 times more than actual
number of operations, similarly the numbers of calls to render_multiple is
about 3 times more, but it might be just previous migrations.
It would also be cool if django would not use inter-operation states at
all or would allow some kind of merged operation, that would not waste
time to compute many states for every small change.
--
Ticket URL: <https://code.djangoproject.com/ticket/24670#comment:2>
Django
migration
Reporter: kurevin | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage:
optimization | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by MarkusH):
Hi kurevin, according to the trace you gave above this looks like #24591
which is about to be committet to master and in a slightly modified form
to stable/1.8.x.
Regarding the backports to 1.7: there are two reasons those changes are
not being backported:
1. in 1.8 we now cache the project state across migrations as much as
possible and due to API changes many patches are not easy to be backported
2. 1.7 only receives security and data loss related fixes.
--
Ticket URL: <https://code.djangoproject.com/ticket/24670#comment:3>
Django
migration
Reporter: kurevin | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Migrations | Version: master
Keywords: migrations | Triage Stage:
optimization | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* status: new => closed
* resolution: => duplicate
Comment:
Closing as a duplicate per Markus's comment. Please open separate tickets
if there are other ideas for optimizations.
--
Ticket URL: <https://code.djangoproject.com/ticket/24670#comment:4>