caching questions
fre...@pythonware.com
and it works like a charm (450+ hits per second on a cheap-ass
server isn't all that bad ;-).
however, there are two things that I haven't been able to sort
out on my own:
- the admin site is cached too, which makes it a bit hard to use.
it might be a middleware ordering problem, but I haven't found a
working combination. any ideas?
- I would like to explicitly remove pages from the cache, based
on the URL. is there some public API for this that I haven't found ?
(I'm using the "stock" 0.91 release, btw)
cheers /F
fre...@pythonware.com
Cheng Zhang
On Mar 1, 2006, at 3:57 PM, fre...@pythonware.com wrote:
>
> anyone ?
I guess guys and gals are busy with PyCon Sprint these couple days,
so they couldn't pay much attention on the mailing list. That's unusual.
Personally I haven't use cache in a production Django site yet, but I
will try to share my understandings.
>
>> however, there are two things that I haven't been able to sort
>> out on my own:
>>
>> - the admin site is cached too, which makes it a bit hard to use.
>> it might be a middleware ordering problem, but I haven't found a
>> working combination. any ideas?
How about running the admin site with another process, say another
mod_python configuration, separated with the main site? This way, you
can have a separate settings for the admin site.
>>
>> - I would like to explicitly remove pages from the cache, based
>> on the URL. is there some public API for this that I haven't found ?
There is a section in the Cache API document called "Controlling
cache: Using Vary headers" (http://www.djangoproject.com/
documentation/cache/#controlling-cache-using-vary-headers). If you
didn't check it out, it might be useful for you, although I didn't
have chance to use it yet.
Cheers
- Cheng Zhang
Malcolm Tredinnick
On Wed, 2006-03-01 at 07:57 +0000, fre...@pythonware.com wrote:
> anyone ?
>
> > however, there are two things that I haven't been able to sort
> > out on my own:
> >
> > - the admin site is cached too, which makes it a bit hard to use.
> > it might be a middleware ordering problem, but I haven't found a
> > working combination. any ideas?
Not going to touch this one; I'm not sure what is going on.
> > - I would like to explicitly remove pages from the cache, based
> > on the URL. is there some public API for this that I haven't found ?
Since you can control caching on a per-view-function basis, is it
possible to set things up so that the URLs you do not wish to cache are
served via a different view than the others (even if the view is just a
wrapper around the common stuff)? If you can organise this, then using
the @cache-control decorator can set the caching mechanism to "private",
which should achieve what you want. This is documented in
docs/cache.txt.
However, maybe I am not understanding what you are asking, because you
are saying "remove pages from the cache" (referring to the Django
implementation?), whereas I am answering "how not to cache pages on an
HTTP level" (referring to the result going out to the user). I am
worried that your question is precise enough that I am not really
answering what is being asked, so feel free to post more clues if you
think that will help.
Regards,
Malcolm
Adrian Holovaty
> - the admin site is cached too, which makes it a bit hard to use.
> it might be a middleware ordering problem, but I haven't found a
> working combination. any ideas?
Ah, you must be using the per-site cache, right? I'm actually not sure
how the per-site cache would interact with the admin site -- the POSTs
would work, but the GETs would probably be cached. Is that what's
happening?
The best solution to this is to use per-view caches rather than the
per-site cache; that way, you have more fine-grained control over
which types of pages get cached.
> - I would like to explicitly remove pages from the cache, based
> on the URL. is there some public API for this that I haven't found ?
It's possible to interpret this question two ways, so I'll answer both:
* If you're wanting to physically delete a page from the cache, use
the low-level cache API:
from django.core.cache import cache
cache.set('views.decorators.cache.cache_page../some/url/.d41d8cd98f00b204e9800998ecf8427e',
None)
Note that the cache key contains "/some/url/", which is the URL that
you're wanting to remove from the cache, and the string
"d41d8cd98f00b204e9800998ecf8427e", which is a hash of "empty"
headers. That comes from django.utils.cache.get_cache_key(), which
looks at all the headers that should be considered with respect to
creating the cache key. "d41d8cd98f00b204e9800998ecf8427e" is the hash
for empty headers. Does this make sense? We should probably have a
simple helper function that clears the cache for a given URL, assuming
no headers.
* If you want to prevent a particular view from being cached (when
using site-wide cache middleware), there currently isn't a way to do
that. The tricky problem there is that it would cause a slight
performance decrease: Currently the cache middleware just looks at
each incoming URL, checks the cache for the URL with that key, and
returns the cached page if the key is found. If we were to allow
certain views to be left out of the cache, the cache middleware would
have to do URL dispatching to find out what the associated view is,
then check the view to see whether it should use the cache, then
finally check the cache. The extra overhead of URL dispatching isn't a
huge deal, but it's still kind of counter to caching. Thoughts on this
are welcome!
Adrian
--
Adrian Holovaty
holovaty.com | djangoproject.com
fre...@pythonware.com
> > - the admin site is cached too, which makes it a bit hard to use.
> > it might be a middleware ordering problem, but I haven't found a
> > working combination. any ideas?
>
> Ah, you must be using the per-site cache, right? I'm actually not sure
> how the per-site cache would interact with the admin site -- the POSTs
> would work, but the GETs would probably be cached. Is that what's
> happening?
yup. the caching docs didn't say anything about this, so I happily
assumed that you'd thought of this already ;-)
> The best solution to this is to use per-view caches rather than the
> per-site cache; that way, you have more fine-grained control over
> which types of pages get cached.
to what extent does all the caching header magic (Last-Modified,
Expires, etc) work when I use cache decorators? do I have to deal
with that myself, or is a
@cache_page(...)
def view(...)
exactly equivalent to a plain view plus a full-site cache?
> > - I would like to explicitly remove pages from the cache, based
> > on the URL. is there some public API for this that I haven't found ?
>
> It's possible to interpret this question two ways, so I'll answer both:
I accidentally sent a clarification to directly to Malcolm (I'm not
entirely google-groups compatible, it seems, and googlegroups
don't like me either, as can be seen from the ratings my posts
usually get), and he's sorted out the basics for me already.
(thanks, Malcolm!)
anyway,
> * If you're wanting to physically delete a page from the cache, use
> the low-level cache API:
this is what I meant to ask.
> from django.core.cache import cache
> cache.set('views.decorators.cache.cache_page../some/url/.d41d8cd98f00b204e9800998ecf8427e',
> None)
>
> Note that the cache key contains "/some/url/", which is the URL that
> you're wanting to remove from the cache, and the string
> "d41d8cd98f00b204e9800998ecf8427e", which is a hash of "empty"
> headers. That comes from django.utils.cache.get_cache_key(), which
> looks at all the headers that should be considered with respect to
> creating the cache key. "d41d8cd98f00b204e9800998ecf8427e" is the hash
> for empty headers. Does this make sense?
absolutely.
>>> md5.md5("").hexdigest()
'd41d8cd98f00b204e9800998ecf8427e'
> We should probably have a simple helper function that clears the cache for
> a given URL, assuming no headers.
that would be the function I was looking for ;-)
a "manage purgecache" command might also be somewhat useful,
at least for people like me who likes to hack on a semi-live server ;-)
> * If you want to prevent a particular view from being cached (when
> using site-wide cache middleware), there currently isn't a way to do
> that. The tricky problem there is that it would cause a slight
> performance decrease: Currently the cache middleware just looks at
> each incoming URL, checks the cache for the URL with that key, and
> returns the cached page if the key is found. If we were to allow
> certain views to be left out of the cache, the cache middleware would
> have to do URL dispatching to find out what the associated view is,
> then check the view to see whether it should use the cache, then
> finally check the cache. The extra overhead of URL dispatching isn't a
> huge deal, but it's still kind of counter to caching. Thoughts on this
> are welcome!
in the application servers I've built, the caching system generally
uses
the following approach:
1. look up incoming requests in the cache
2. if not found, do a full request processing
3. if the resulting response is flagged as uncachable, don't update
the
cache
in other words, cachability is a property of the response, not the
view.
maybe the existing cache decorators can be used to implement step 3
today ?
(the docs are not clear enough on the relation between the cache
system, the caching middleware, and the cache decorators for me
to be able to figure this out on my own... do I have to read the
source ? ;-)
</F>
