Reduce start up time for manage.py

[chj]

Hi,

I'm having a quite large Django project with dozens of apps and many reusable third-party libraries. All the custom management commands (being used with manage.py <command>) take a long time to be executed, since the start up sequence takes between 3 and 5 seconds. Using the cProfile module I was able to see that quite a lot of function calls are executed and it looks like that every app is loaded, including its libraries and such before the actual management command is executed. Is there any common way to speed that up (e.g. by not loading apps/libraries that are not required for a specific management command)? Django 1.7+ is used on Python 3.4.

Output from `python3 -m cProfile manage.py` (truncated): 1100697 function calls (1064507 primitive calls) in 5.172 seconds

[Collin Anderson]

Hi,

Many people would recommend against this, but if you can put the imports for your heavy 3rd party libraries inside functions and methods, that allow them to be loaded only if needed.

I would also recommend in general to simply have fewer libraries and apps (easier said than done :)

Also, is it slow only for the first load, or are repeated loads slow too?

I hope you aren't doing database queries or other expensive calls on startup.

Try putting this at the top of your manage.py to see more of what's going on

import logging; logging.basicConfig(level=logging.DEBUG)

Collin

If you want some benchmarks, here are some of my websites

221243 function calls (214987 primitive calls) in 0.394 seconds

448659 function calls (437433 primitive calls) in 0.611 seconds 

698349 function calls (687730 primitive calls) in 0.749 seconds

[Avraham Serour]

I believe putting imports inside functions would slow down execution, it will make the import everytime the function is executed

one could make something like a lazy import, but you would be trading slow startup for slow requests

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/0ac3ca3d-78a5-49bf-be96-05bc0017bf40%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daniel Roseman]

On Friday, 30 January 2015 14:01:17 UTC, Avraham Serour wrote:

I believe putting imports inside functions would slow down execution, it will make the import everytime the function is executed

one could make something like a lazy import, but you would be trading slow startup for slow requests

That is not true. Python imports are always only executed once: if the module already exists in `sys.modules`, it is not loaded again. 

--

DR.

[dalore]

One run manage with a different settings file which has a reduced INSTALLED_APPS