Models.py not loaded at server startup ?????

89 views
Skip to first unread message

Pakal

unread,
Dec 12, 2010, 4:40:23 PM12/12/10
to Django developers
Hello

Having spent my time developing with django's dev server, I was pretty
used to having all models (re)validated when the server reloaded.

However, when I connected the codebase to an apache/mod_python
instance, I had the surprise of seeing the miscellaneous models.py
INSTALLED_APPS didn't get loaded on startup - blatantly waiting for a
request to trigger their import.

That's very embarassing, because we use models.py as a place for
startup codes, and we like to have all models automatically loaded at
startup to quickly detect deployment errors (missing modules etc.).

From all the threads browsed on the web, I got that actually only
urls.py or middlewares should be used for startup code - no mention is
done of models.py. But why then is everyone using models.py to connect
signals and other startup tasks ? It's a real fault, if that model
isn't required by a particular view, signals wont be setup, and the
workflow will be broken.

Why, then, isn't it specified that all models.py files should be
loaded by each starting worker ? That would solve the whole problem
and hidden errors around startup code like signals and startup checks.

Thanks for your attention, regards,
Pakal

Carl Meyer

unread,
Dec 13, 2010, 9:47:19 AM12/13/10
to Django developers
Hi,

On Dec 12, 4:40 pm, Pakal <chambon.pas...@gmail.com> wrote:
> Why, then, isn't it specified that all models.py files should be
> loaded by each starting worker ? That would solve the whole problem
> and hidden errors around startup code like signals and startup checks.

This is a real issue for me as well; not necessarily that models.py
isn't loaded at startup time, but the crucial difference in behavior
between runserver and a live deployment in terms of when apps' models
are imported, due to runserver performing model validation and thus
importing all models immediately. I have more than once seen code that
worked perfectly under runserver fail in subtle ways on real
deployment for this reason. (In fact, the most popular Django search
solution, haystack, is particularly susceptible to problems of this
type, as it does some dynamic importing immediately when its models.py
is imported).

So this is certainly on my radar screen as something I'd like to look
at in the 1.4 timeframe. I'm not sure yet what a good solution would
look like, and I'm not entirely convinced that importing all models.py
at startup time is the right answer. In any case, the Google Summer of
Code work by Arthur Koziel on app-classes will impact this area, and
is scheduled for merge in the 1.4 cycle, so I'm kind of waiting for
that to land before looking at this closely.

Carl

Harro

unread,
Dec 13, 2010, 10:54:11 AM12/13/10
to Django developers
Is this mod_python specific or does it also happen with mod_wsgi or
gunicorn?

Pakal

unread,
Dec 13, 2010, 2:01:30 PM12/13/10
to Django developers
I've never used these handlers yet, but from what I've browsed from
django's code, it seems low level handlers have nothing to import
models, so I guess the result would be the same.
I guess I'll manually import all my INSTALLED_APPS models from within
my settings.py or urls.py, for the moment.

But I still don't understand : where are we supposed to connect
signals, then ? The signaling framework only makes sense if there is a
clear startup phase in the server code, else it's just a nasty (and
silent) trap for programmers, isn't it ?

Regards,
Pakal

Michael

unread,
Sep 1, 2012, 12:23:25 PM9/1/12
to django-d...@googlegroups.com
Hi,

Is there a "clean" solution nowadays?

I am making an app that should run some specific code after the whole project is loaded.
I was thinking about a thread started in the models.py file of my app that will try X times to run the code, something like that:
success = False
attempts = 0
while not success and attempts < 10:
    attempts += 1
    try:
         run() # raise Exception if it fails
         success = True
   except:
         pass

if attempts == 10 and not success:
  # log or do something


What do you think? Any ideas?

Thanks.

Michael

Yo-Yo Ma

unread,
Sep 2, 2012, 9:20:44 AM9/2/12
to django-d...@googlegroups.com
Another good reason for model loading at startup is a good idea; without it, models.get_models becomes a lie.

Russell Keith-Magee

unread,
Sep 2, 2012, 11:00:26 AM9/2/12
to django-d...@googlegroups.com
… and this is exactly why the app-refactor has been a topic of
discussion for several years. Hopefully we'll get a chance to thrash
out some of the details this week at DjangoCon US; I know Preston (who
has been working on the patch of late) is keen to discuss what he's
done.

Yours,
Russ Magee %-)

On Sun, Sep 2, 2012 at 9:20 PM, Yo-Yo Ma <baxters...@gmail.com> wrote:
> Another good reason for model loading at startup is a good idea; without it, models.get_models becomes a lie.
>
> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/bDjcdKrvk9MJ.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
>
Reply all
Reply to author
Forward
0 new messages