Migrations processing time of reload_model
43 views
Skip to first unread message
Mounir Messelmeni
Nov 17, 2015, 5:27:25 AM11/17/15
to Django developers (Contributions to Django itself)
Hi,
On one of my projects, I'm having processing time trouble to generate or run migrations even for a single app, this can take up to 3 minutes, after debugging I found that the function reload_model inside django.db.migrations.state.py take a lot of time, and this one is called when evaluating every migrations operation (AddField, AlterField ...)
The question is, I'm I supposed to have something wrong on my models or is this a common behaviour of Django's migrations?
Here a stats from cProfile for the calls for realod_model method:
353318 function calls (352524 primitive calls) in 2.859 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.000 0.000 <string>:8(__new__)
1 0.000 0.000 0.000 0.000 AES.py:55(__init__)
1 0.000 0.000 0.000 0.000 AES.py:61(new)
1 0.000 0.000 0.000 0.000 UserDict.py:26(clear)
31 0.001 0.000 0.005 0.000 __init__.py:1005(__init__)
31 0.000 0.000 0.014 0.000 __init__.py:1027(deconstruct)
155 0.005 0.000 0.025 0.000 __init__.py:1080(__init__)
250 0.003 0.000 0.047 0.000 __init__.py:112(__iter__)
138 0.002 0.000 0.020 0.000 __init__.py:1195(__init__)
138 0.002 0.000 0.057 0.000 __init__.py:1246(deconstruct)
138 0.005 0.000 0.039 0.000 __init__.py:1298(contribute_to_class)
693 0.053 0.000 0.087 0.000 __init__.py:137(__init__)
55 0.001 0.000 0.009 0.000 __init__.py:1502(__init__)
55 0.001 0.000 0.026 0.000 __init__.py:1582(deconstruct)
5 0.000 0.000 0.001 0.000 __init__.py:1707(__init__)
5 0.000 0.000 0.002 0.000 __init__.py:1712(deconstruct)
676 0.009 0.000 0.021 0.000 __init__.py:20(import_module)
2 0.000 0.000 0.000 0.000 __init__.py:2033(__init__)
2 0.000 0.000 0.001 0.000 __init__.py:2038(deconstruct)
4 0.000 0.000 0.001 0.000 __init__.py:2118(__init__)
4 0.000 0.000 0.001 0.000 __init__.py:2125(deconstruct)
249 0.007 0.000 0.011 0.000 __init__.py:22(<lambda>)
5 0.000 0.000 0.001 0.000 __init__.py:2310(__init__)
5 0.000 0.000 0.002 0.000 __init__.py:2314(deconstruct)
1 0.000 0.000 0.000 0.000 __init__.py:233(__len__)
2 0.000 0.000 0.000 0.000 __init__.py:245(__bool__)
70 0.001 0.000 0.003 0.000 __init__.py:35(__getattr__)
1 0.000 0.000 0.000 0.000 __init__.py:35(get_option)
35 0.001 0.000 0.008 0.000 __init__.py:352(__init__)
666 0.128 0.000 0.310 0.000 __init__.py:358(deconstruct)
70 0.001 0.000 0.035 0.000 __init__.py:369(deconstruct)
35 0.001 0.000 0.007 0.000 __init__.py:388(__init__)
35 0.000 0.000 0.019 0.001 __init__.py:403(deconstruct)
4 0.000 0.000 0.000 0.000 __init__.py:45(countries)
15 0.000 0.000 0.000 0.000 __init__.py:455(__eq__)
1801 0.014 0.000 0.021 0.000 __init__.py:46(__getattr__)
1761 0.011 0.000 0.019 0.000 __init__.py:461(__lt__)
158 0.001 0.000 0.001 0.000 __init__.py:528(validators)
1 0.000 0.000 0.000 0.000 __init__.py:64(configured)
693 0.007 0.000 0.019 0.000 __init__.py:647(set_attributes_from_name)
693 0.014 0.000 0.166 0.000 __init__.py:655(contribute_to_class)
598 0.002 0.000 0.002 0.000 __init__.py:666(get_attname)
598 0.007 0.000 0.009 0.000 __init__.py:669(get_attname_column)
95 0.001 0.000 0.001 0.000 __init__.py:674(get_cache_name)
249 0.002 0.000 0.022 0.000 __init__.py:83(ugettext)
693 0.006 0.000 0.034 0.000 __init__.py:854(_get_choices)
72 0.002 0.000 0.015 0.000 __init__.py:929(__init__)
63 0.001 0.000 0.032 0.001 __init__.py:951(deconstruct)
72 0.001 0.000 0.026 0.000 __init__.py:987(contribute_to_class)
73 0.003 0.000 0.007 0.000 _abcoll.py:526(update)
2271 0.015 0.000 0.015 0.000 _weakrefset.py:70(__contains__)
1432 0.021 0.000 0.040 0.000 abc.py:128(__instancecheck__)
73 0.001 0.000 0.001 0.000 abc.py:148(__subclasscheck__)
73 0.001 0.000 0.001 0.000 base.py:146(<genexpr>)
73 0.001 0.000 0.001 0.000 base.py:155(<genexpr>)
73 0.001 0.000 0.001 0.000 base.py:197(<setcomp>)
990/927 0.018 0.000 0.461 0.000 base.py:321(add_to_class)
73 0.006 0.000 0.117 0.002 base.py:328(_prepare)
757 0.002 0.000 0.002 0.000 base.py:361(<genexpr>)
146 0.013 0.000 0.013 0.000 base.py:44(subclass_exception)
73/64 0.021 0.000 1.762 0.028 base.py:72(__new__)
1 0.000 0.000 0.000 0.000 blockalgo.py:124(_getParameter)
1 0.000 0.000 0.000 0.000 blockalgo.py:136(__init__)
7245 0.175 0.000 0.306 0.000 collections.py:104(values)
73 0.005 0.000 0.014 0.000 collections.py:38(__init__)
146 0.001 0.000 0.001 0.000 collections.py:54(__setitem__)
146 0.001 0.000 0.001 0.000 collections.py:64(__delitem__)
38439 0.132 0.000 0.132 0.000 collections.py:73(__iter__)
1 0.000 0.000 0.000 0.000 conf.py:10(__getattribute__)
7053 0.024 0.000 0.024 0.000 config.py:143(check_models_ready)
2 0.000 0.000 0.000 0.000 config.py:151(get_model)
25524 0.232 0.000 0.535 0.000 config.py:164(get_models)
387 0.003 0.000 0.021 0.000 datastructures.py:486(__new__)
316/158 0.002 0.000 0.003 0.000 deconstruct.py:19(__new__)
75 0.001 0.000 0.001 0.000 dispatcher.py:14(_make_id)
73 0.007 0.000 0.044 0.001 dispatcher.py:166(send)
74 0.000 0.000 0.000 0.000 dispatcher.py:234(_clear_dead_receivers)
73 0.012 0.000 0.022 0.000 dispatcher.py:245(_live_receivers)
1 0.000 0.000 0.000 0.000 dispatcher.py:54(connect)
666 0.004 0.000 0.006 0.000 encoding.py:63(is_protected_type)
948 0.012 0.000 0.048 0.000 encoding.py:72(force_text)
1 0.000 0.000 0.000 0.000 fields.py:104(__init__)
1 0.000 0.000 0.000 0.000 fields.py:144(__init__)
1 0.000 0.000 0.000 0.000 fields.py:158(contribute_to_class)
1 0.000 0.000 0.048 0.048 fields.py:179(deconstruct)
2 0.000 0.000 0.000 0.000 fields.py:19(__init__)
1 0.000 0.000 0.000 0.000 fields.py:23(__init__)
1 0.000 0.000 0.000 0.000 fields.py:36(__init__)
1 0.000 0.000 0.000 0.000 fields.py:60(__init__)
3 0.000 0.000 0.000 0.000 files.py:170(__init__)
3 0.000 0.000 0.000 0.000 files.py:240(__init__)
3 0.000 0.000 0.001 0.000 files.py:284(deconstruct)
3 0.000 0.000 0.000 0.000 files.py:317(contribute_to_class)
1 0.000 0.000 0.000 0.000 files.py:390(__init__)
1 0.000 0.000 0.000 0.000 files.py:416(deconstruct)
1 0.000 0.000 0.000 0.000 files.py:424(contribute_to_class)
24 0.007 0.000 0.013 0.001 functional.py:102(__prepare_class__)
249 0.002 0.000 0.023 0.000 functional.py:139(__text_cast)
270 0.001 0.000 0.001 0.000 functional.py:14(curry)
24 0.000 0.000 0.014 0.001 functional.py:188(__wrapper__)
437/336 0.005 0.000 0.065 0.000 functional.py:56(__get__)
24 0.000 0.000 0.014 0.001 functional.py:89(__init__)
27 0.000 0.000 0.001 0.000 functools.py:56(<lambda>)
249 0.001 0.000 0.001 0.000 gettext.py:217(ugettext)
1 0.000 0.000 0.000 0.000 inspect.py:142(isfunction)
1 0.000 0.000 0.000 0.000 inspect.py:209(iscode)
1 0.000 0.000 0.000 0.000 inspect.py:47(func_accepts_kwargs)
990 0.007 0.000 0.012 0.000 inspect.py:59(isclass)
1 0.000 0.000 0.000 0.000 inspect.py:67(ismethod)
1 0.000 0.000 0.000 0.000 inspect.py:744(getargs)
1 0.000 0.000 0.000 0.000 inspect.py:804(getargspec)
74 0.002 0.000 0.012 0.000 lru_cache.py:160(cache_clear)
161 0.002 0.000 0.003 0.000 lru_cache.py:19(__init__)
282 0.001 0.000 0.001 0.000 lru_cache.py:23(__hash__)
161 0.004 0.000 0.009 0.000 lru_cache.py:26(_make_key)
161 0.005 0.000 0.759 0.005 lru_cache.py:94(wrapper)
73 0.004 0.000 0.019 0.000 manager.py:11(ensure_default_manager)
78 0.002 0.000 0.004 0.000 manager.py:157(contribute_to_class)
78 0.002 0.000 0.002 0.000 manager.py:179(_set_creation_counter)
718 0.002 0.000 0.002 0.000 manager.py:25(<genexpr>)
78 0.000 0.000 0.000 0.000 manager.py:250(__init__)
68 0.000 0.000 0.000 0.000 manager.py:253(__get__)
78 0.001 0.000 0.001 0.000 manager.py:62(__new__)
78 0.001 0.000 0.003 0.000 manager.py:68(__init__)
5 0.000 0.000 0.000 0.000 manager.py:83(deconstruct)
3 0.000 0.000 0.000 0.000 modelfields.py:25(__init__)
3 0.000 0.000 0.000 0.000 modelfields.py:46(__init__)
3 0.000 0.000 0.001 0.000 modelfields.py:68(contribute_to_class)
671 0.012 0.000 0.040 0.000 module_loading.py:15(import_string)
70 0.000 0.000 0.000 0.000 operations.py:194(max_name_length)
73 0.024 0.000 0.072 0.001 options.py:197(contribute_to_class)
73 0.001 0.000 0.006 0.000 options.py:252(_prepare)
693 0.015 0.000 0.099 0.000 options.py:289(add_field)
684 0.003 0.000 0.003 0.000 options.py:319(setup_pk)
34485 0.111 0.000 0.122 0.000 options.py:349(swapped)
73 0.002 0.000 0.034 0.000 options.py:375(fields)
693 0.002 0.000 0.002 0.000 options.py:392(<lambda>)
684 0.002 0.000 0.002 0.000 options.py:393(<lambda>)
684 0.002 0.000 0.003 0.000 options.py:394(<lambda>)
757 0.007 0.000 0.014 0.000 options.py:399(<genexpr>)
146 0.001 0.000 0.002 0.000 options.py:60(normalize_together)
1 0.003 0.003 0.026 0.026 options.py:672(_populate_directed_relation_graph)
285 0.002 0.000 0.004 0.000 options.py:688(<genexpr>)
1 0.000 0.000 0.026 0.026 options.py:707(_relation_tree)
10433 0.430 0.000 0.766 0.000 options.py:711(_expire_cache)
75 0.001 0.000 0.041 0.001 options.py:728(get_fields)
316/148 0.012 0.000 0.053 0.000 options.py:742(_get_fields)
18 0.000 0.000 0.000 0.000 options.py:75(<genexpr>)
3256 0.009 0.000 0.009 0.000 options.py:811(<genexpr>)
387 0.005 0.000 0.027 0.000 options.py:82(make_immutable_fields_list)
314 0.001 0.000 0.001 0.000 options.py:820(<genexpr>)
73 0.004 0.000 0.019 0.000 options.py:92(__init__)
73 0.001 0.000 0.001 0.000 re.py:251(_compile_repl)
73 0.001 0.000 0.002 0.000 re.py:271(_subx)
135 0.001 0.000 0.005 0.000 re.py:277(filter)
2 0.000 0.000 0.000 0.000 registry.py:119(check_apps_ready)
224 0.001 0.000 0.001 0.000 registry.py:126(check_models_ready)
2 0.000 0.000 0.000 0.000 registry.py:140(get_app_config)
121 0.134 0.001 0.742 0.006 registry.py:153(get_models)
2 0.000 0.000 0.000 0.000 registry.py:186(get_model)
73 0.019 0.000 0.069 0.001 registry.py:234(get_containing_app_config)
110 0.002 0.000 0.003 0.000 registry.py:255(get_registered_model)
74 0.055 0.001 1.201 0.016 registry.py:334(clear_cache)
101 0.001 0.000 0.001 0.000 related.py:107(related_model)
9 0.000 0.000 0.000 0.000 related.py:1151(__init__)
9 0.000 0.000 0.000 0.000 related.py:1209(__init__)
104 0.001 0.000 0.001 0.000 related.py:1272(__init__)
104 0.000 0.000 0.000 0.000 related.py:1300(related_model)
95 0.000 0.000 0.000 0.000 related.py:1352(is_hidden)
98 0.001 0.000 0.001 0.000 related.py:1372(get_accessor_name)
12 0.000 0.000 0.000 0.000 related.py:1394(get_cache_name)
95 0.004 0.000 0.005 0.000 related.py:1402(__init__)
95 0.001 0.000 0.001 0.000 related.py:1425(set_field_name)
12 0.000 0.000 0.003 0.000 related.py:1430(__init__)
9 0.000 0.000 0.000 0.000 related.py:1439(__init__)
18 0.000 0.000 0.000 0.000 related.py:1455(is_hidden)
95 0.002 0.000 0.018 0.000 related.py:1487(__init__)
77 0.004 0.000 0.491 0.006 related.py:1559(deconstruct)
95 0.003 0.000 0.056 0.001 related.py:1774(contribute_to_class)
95 0.004 0.000 0.007 0.000 related.py:1778(contribute_to_related_class)
95 0.006 0.000 0.031 0.000 related.py:1803(__init__)
77 0.002 0.000 0.494 0.006 related.py:1875(deconstruct)
95 0.001 0.000 0.001 0.000 related.py:1927(get_attname)
95 0.001 0.000 0.001 0.000 related.py:1930(get_attname_column)
95 0.001 0.000 0.008 0.000 related.py:1965(contribute_to_related_class)
12 0.001 0.000 0.008 0.001 related.py:2033(__init__)
12 0.000 0.000 0.072 0.006 related.py:2037(deconstruct)
9 0.001 0.000 0.162 0.018 related.py:2059(create_many_to_many_intermediary_model)
9 0.000 0.000 0.000 0.000 related.py:2066(set_managed)
9 0.001 0.000 0.002 0.000 related.py:2123(__init__)
9 0.000 0.000 0.058 0.006 related.py:2394(deconstruct)
9 0.000 0.000 0.001 0.000 related.py:2462(_get_m2m_db_table)
9 0.001 0.000 0.165 0.018 related.py:2529(contribute_to_class)
9 0.001 0.000 0.001 0.000 related.py:2567(contribute_to_related_class)
9 0.000 0.000 0.000 0.000 related.py:2585(set_attributes_from_rel)
104 0.005 0.000 0.054 0.001 related.py:293(contribute_to_class)
110 0.005 0.000 0.019 0.000 related.py:30(add_lazy_relation)
101 0.001 0.000 0.013 0.000 related.py:310(resolve_related_class)
86 0.106 0.001 0.509 0.006 related.py:317(swappable_setting)
95 0.001 0.000 0.001 0.000 related.py:341(set_attributes_from_rel)
104 0.001 0.000 0.012 0.000 related.py:354(do_related_class)
12 0.000 0.000 0.000 0.000 related.py:402(__init__)
95 0.001 0.000 0.002 0.000 related.py:526(__init__)
65 0.000 0.000 0.000 0.000 related.py:805(__init__)
73 0.001 0.000 0.007 0.000 related.py:89(do_pending_lookups)
511 0.005 0.000 0.007 0.000 signals.py:20(_resolve_references)
1 0.000 0.000 0.000 0.000 signals.py:33(connect)
135 0.002 0.000 0.004 0.000 sre_parse.py:790(expand_template)
1 0.001 0.001 2.798 2.798 state.py:256(render_multiple)
1 0.000 0.000 0.000 0.000 state.py:26(_get_app_label_and_model_name)
73 0.002 0.000 1.196 0.016 state.py:289(register_model)
73 0.001 0.000 0.002 0.000 state.py:297(unregister_model)
2 0.002 0.001 0.049 0.025 state.py:34(get_related_models_recursive)
75 0.005 0.000 0.047 0.001 state.py:45(_related_models)
730 0.025 0.000 1.017 0.001 state.py:492(construct_fields)
69 0.001 0.000 0.002 0.000 state.py:499(construct_managers)
5 0.000 0.000 0.000 0.000 state.py:502(<lambda>)
64 0.012 0.000 2.797 0.044 state.py:524(render)
155 0.001 0.000 0.002 0.000 state.py:533(<genexpr>)
1 0.001 0.001 2.859 2.859 state.py:97(_reload_model)
2 0.000 0.000 0.000 0.000 subclassing.py:35(__init__)
2 0.000 0.000 0.000 0.000 subclassing.py:56(contribute_to_class)
73 0.002 0.000 0.015 0.000 text.py:455(camel_case_to_spaces)
74 0.003 0.000 0.004 0.000 threading.py:146(acquire)
74 0.001 0.000 0.002 0.000 threading.py:186(release)
74 0.001 0.000 0.003 0.000 threading.py:214(__exit__)
148 0.000 0.000 0.000 0.000 threading.py:63(_note)
249 0.012 0.000 0.018 0.000 trans_real.py:287(do_translate)
249 0.002 0.000 0.020 0.000 trans_real.py:326(ugettext)
70 0.000 0.000 0.001 0.000 utils.py:178(truncate_name)
70 0.001 0.000 0.001 0.000 utils.py:234(__getitem__)
1 0.000 0.000 0.000 0.000 validators.py:119(__init__)
158 0.001 0.000 0.001 0.000 validators.py:275(__init__)
1 0.000 0.000 0.000 0.000 weakref_backports.py:25(__new__)
511 0.003 0.000 0.003 0.000 weakref_backports.py:47(__call__)
15 0.000 0.000 0.000 0.000 writer.py:35(__new__)
15 0.000 0.000 0.000 0.000 writer.py:38(__init__)
1 0.000 0.000 0.000 0.000 {Crypto.Cipher._AES.new}
676 0.010 0.000 0.010 0.000 {__import__}
693 0.010 0.000 0.029 0.000 {_bisect.bisect}
68 0.002 0.000 0.004 0.000 {any}
714 0.012 0.000 0.026 0.000 {built-in method __new__ of type object at 0x91f8e0}
4 0.000 0.000 0.000 0.000 {callable}
81652 0.280 0.000 0.280 0.000 {delattr}
22415/22120 0.079 0.000 0.081 0.000 {getattr}
5093 0.019 0.000 0.019 0.000 {hasattr}
161 0.001 0.000 0.001 0.000 {hash}
76 0.000 0.000 0.000 0.000 {id}
8617 0.041 0.000 0.082 0.000 {isinstance}
9 0.000 0.000 0.000 0.000 {iter}
705 0.002 0.000 0.002 0.000 {len}
75 0.000 0.000 0.000 0.000 {method '__subclasses__' of 'type' objects}
74 0.000 0.000 0.000 0.000 {method 'acquire' of 'thread.lock' objects}
391 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
2339 0.007 0.000 0.007 0.000 {method 'append' of 'list' objects}
75 0.003 0.000 0.003 0.000 {method 'clear' of 'dict' objects}
1 0.000 0.000 0.000 0.000 {method 'copy' of 'dict' objects}
73 0.001 0.000 0.001 0.000 {method 'copy' of 'dictproxy' objects}
249 0.001 0.000 0.001 0.000 {method 'decode' of 'str' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
249 0.002 0.000 0.002 0.000 {method 'encode' of 'unicode' objects}
28020 0.090 0.000 0.100 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 {method 'format' of 'unicode' objects}
13949 0.038 0.000 0.038 0.000 {method 'get' of 'dict' objects}
135 0.000 0.000 0.000 0.000 {method 'group' of '_sre.SRE_Match' objects}
693 0.003 0.000 0.003 0.000 {method 'insert' of 'list' objects}
974 0.005 0.000 0.005 0.000 {method 'items' of 'dict' objects}
135 0.001 0.000 0.001 0.000 {method 'join' of 'str' objects}
73 0.003 0.000 0.005 0.000 {method 'join' of 'unicode' objects}
72 0.000 0.000 0.000 0.000 {method 'keys' of 'dictproxy' objects}
111 0.000 0.000 0.000 0.000 {method 'lower' of 'str' objects}
508 0.003 0.000 0.003 0.000 {method 'lower' of 'unicode' objects}
29 0.000 0.000 0.000 0.000 {method 'mro' of 'type' objects}
1880 0.006 0.000 0.006 0.000 {method 'pop' of 'dict' objects}
74 0.000 0.000 0.000 0.000 {method 'release' of 'thread.lock' objects}
9 0.000 0.000 0.000 0.000 {method 'replace' of 'str' objects}
1341 0.005 0.000 0.005 0.000 {method 'replace' of 'unicode' objects}
5 0.000 0.000 0.000 0.000 {method 'rsplit' of 'str' objects}
666 0.004 0.000 0.004 0.000 {method 'rsplit' of 'unicode' objects}
209 0.001 0.000 0.001 0.000 {method 'setdefault' of 'dict' objects}
344 0.003 0.000 0.003 0.000 {method 'split' of 'str' objects}
63 0.002 0.000 0.002 0.000 {method 'split' of 'unicode' objects}
302 0.001 0.000 0.001 0.000 {method 'startswith' of 'str' objects}
8427 0.028 0.000 0.028 0.000 {method 'startswith' of 'unicode' objects}
73 0.000 0.000 0.000 0.000 {method 'strip' of 'unicode' objects}
73 0.005 0.000 0.012 0.000 {method 'sub' of '_sre.SRE_Pattern' objects}
4182 0.017 0.000 0.019 0.000 {method 'update' of 'dict' objects}
2 0.000 0.000 0.000 0.000 {method 'update' of 'set' objects}
1 0.000 0.000 0.000 0.000 {method 'upper' of 'str' objects}
1 0.000 0.000 0.000 0.000 {method 'upper' of 'unicode' objects}
9 0.000 0.000 0.000 0.000 {next}
1 0.000 0.000 0.000 0.000 {range}
982 0.006 0.000 0.006 0.000 {setattr}
226 0.004 0.000 0.015 0.000 {sorted}
148 0.000 0.000 0.000 0.000 {thread.get_ident}
249 0.001 0.000 0.001 0.000 {unicodedata.normalize}
Regards,
Mounir
Reply all
Reply to author
Forward
0 new messages