Where/how to load global static data?
Lars Ruoff
i'm using Django the first time for a browser game project.
I will need to have some globally accessible, static data like string
tables for every view.
These would be loaded from XML files.
For the moment all i have figured out with Django is that my code
will
be called via callbacks for the view when accessing a particular URL.
Obviously, i don't want to reload that data with *each* access to a
view.
I'd like to know where to put the code that reads these files and
initialzes the global data once and for all at server startup.
I.e. i'd like to have that data in memory and accessible from any
view
during the lifetime of the server, so that it needs NOT to be loaded
for any access to the site/view.
Hope the question is clear.
Regards,
Lars
kenneth gonsalves
> I'd like to know where to put the code that reads these files and
> initialzes the global data once and for all at server startup.
> I.e. i'd like to have that data in memory and accessible from any
> view
> during the lifetime of the server, so that it needs NOT to be loaded
> for any access to the site/view.
>
>
I may be wrong, but AFAIK django is not a server - it is a framework
that runs on a web server, so every new request starts a new instance of
your site. You could use cacheing to serve static data - but this is not
like pre-loading data like one would do with zope for example.
--
regards
Kenneth Gonsalves
Lars Ruoff
> I may be wrong, but AFAIK django is not a server - it is a framework
> that runs on a web server, so every new request starts a new instance of
> your site. You could use cacheing to serve static data - but this is not
> like pre-loading data like one would do with zope for example.
> --
> regards
> Kenneth Gonsalves
the question is not about serving static data (like HTML, images
etc.), but having global python variables initialized once, for all
instances of the site, that is, so they can be used from within each
view.
All pages should be able to access these global objects and they
should be loaded only once in memory.
The problem is indeed with the "start a new instance of the site".
Isn't there a way to pass constant data between instances?
It probably would require a callback that django calls when it is
invoked for the first time.
does this exist?
kenneth gonsalves
> the question is not about serving static data (like HTML, images
> etc.), but having global python variables initialized once, for all
> instances of the site, that is, so they can be used from within each
> view.
this is precisely the point I addressed in my reply. Every request to
django starts a new instance of django - and you have to initialise the
variables again - cacheing may improve the performance. Django is not a
server.
--
regards
Kenneth Gonsalves
Lars Ruoff
So as i understand, "new instance of Django" implies "new instance of
Python interpreter", right?
Then effectively there is no way of saving objects between instances. :
(
So how would you guys implement a (constant, possibly large) string
table that would be used in nearly every view?
regards,
Lars
Tom Evans
This is not true, please do not repeat it. Of course there are things
that are setup and torn down as part of the request/response cycle,
but there are many parts that are not.
OP: I have a couple of solutions. One of the first things the server
does is import your settings, so if you load and parse your static
data at that point, then you will have access to it from that point
onwards.
A second solution, somewhat less hacky, but not actually achieving
what you wish, is to have a function which loads, parses and returns
the data, and memoize the result. Further calls to the function,
within the same request or not, will not require the data to be loaded
again.
I use the second approach to handle static XML files that must be
parsed and loaded. One benefit of the second approach is that you can
memoize based upon a test. I use this so that as soon as the file on
disk has changed, the contents will be reloaded the next time the
function is called. The data is both static and dynamic!
Cheers
Tom
Lars Ruoff
That magnificantly answers my question. :-)
One last thing:
"... a function which loads, parses and returns
Python interpreter environment even for different requests to the
site?
Tom Evans
No; that is highly unlikely, unless using a threaded model to run
Django*. You would have a separate cache per interpreter process.
Ideally, you would have a hook for server_init and hook for
child_init, just like something like Apache httpd, to allow you to
load data before forking. I believe something like this is planned, or
at least desired!
Cheers
Tom
* Whether you run threaded or non threaded is entirely at the
deployer's choice. See [1] for a detailed explanation of how mod_wsgi
will run your site, depending on how you configure it. So if
deployment is not under your control, you shouldn't use methods that
rely on running a threaded model.
[1] http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading
bruno desthuilliers
>
> OP: I have a couple of solutions. One of the first things the server
> does is import your settings, so if you load and parse your static
> data at that point, then you will have access to it from that point
> onwards.
executable code as possible.
A memoized function seems like the best solution here. Just for the
record, the usual way to have code executed at process startup is to
call this code from within one of your app's models.py - as theses
modules are garanteed to be imported by django at process startup (but
used to be executed twice when running anything from manage.py which
can be a PITA - this seems to have beed fixed the recent 1.4 release).
Tom Evans
<bruno.des...@gmail.com> wrote:
> On Apr 5, 3:11 pm, Tom Evans <tevans...@googlemail.com> wrote:
>>
>> OP: I have a couple of solutions. One of the first things the server
>> does is import your settings, so if you load and parse your static
>> data at that point, then you will have access to it from that point
>> onwards.
>
> Please dont. Settings are, well, settings, and should contain as few
> executable code as possible.
>
I would disagree. If the running of your application depends upon
static content loaded from disk, it may be best to load it in
settings.py. If the code fails, your server fails to start (fail
fast).
In this particular place, it may make sense. The OP is talking about
loading static XML files, parsing them into a Python structure, and
making them available for all views, which require them to work.
As an example, say you use shared hosting, and your host writes your
DB credentials to an XML file in your webroot, readable only by the
webserver. Would it be wrong to parse that XML file in settings.py, in
order to produce the settings.DATABASES setting? Clearly not.
Django deliberately makes this path available to developers -
otherwise settings may as well come from a YAML file. It is up to you
to determine whether it is right or wrong for your particular project.
Cheers
Tom