some thoughts about caching in pylons
tor...@gmx.de
hi...
this is just a small braindump of things i noticed with the caching
infrastructure in pylons.
my app is dynamically rendering images using cairo, which i write into
the response with:
response.content_type="image/png"
surface.write_to_png( response )
i did not return anything, because it was working fine.
so when this was working, i wanted to have the images cached.
@beaker_cache() did not work until i returned response.content
from my action.
i also noticed that etag_cache() does not work when @beaker_cache is in
place. (clearly the function is never executed)
it seems to be necessary to make etag_cache() a decorator too, where the
key may be a callable.
i am a bit undecided, if i prefer just one decorator handling this
caching stuff.
another requirement of my app is that most data is invalidated at fixed
intervals( its a game where a "tick" is happening every 15minutes )
( tick is similar to a turn. between ticks you only edit orders, which
the game instances will execute, when the tick happens )
on irc it was pointed out, that most of this caching is not pylons
business. and i should be using a caching reverse proxy, and be all set.
what is the point in this caching infrastructure then anyways ?
however some of these images should only be seen by the user who owns them.
so one solution would be to include a hash in the url.
to round this up:
i am pretty surprised how easy it is to generate some cairo images with
pylons on the fly, but managing the caching of them is a bit more
involving.
it would be nice to have a little chat with the guy who is designing the
new caching stuff. my experience with webapps is pretty limited, i just
got fascinated with pylons 2 weeks ago, and started implementing a game
engine.
and i dont see caching proxies and stuff in my great picture.
--
torben Hohn
Mike Orr
>
> my app is dynamically rendering images using cairo, which i write into
> the response with:
>
> response.content_type="image/png"
> surface.write_to_png( response )
>
> i did not return anything, because it was working fine.
> so when this was working, i wanted to have the images cached.
>
> @beaker_cache() did not work until i returned response.content
> from my action.
The decorator caches the return value, so that makes sense. It
doesn't look inside the response to see what you've written to it.
> i also noticed that etag_cache() does not work when @beaker_cache is in
> place. (clearly the function is never executed)
>
> it seems to be necessary to make etag_cache() a decorator too, where the
> key may be a callable.
I never could get etag_cache to work.
> on irc it was pointed out, that most of this caching is not pylons
> business. and i should be using a caching reverse proxy, and be all set.
> what is the point in this caching infrastructure then anyways ?
That's a rather one-size-fits-all statement. If you have a lot of
pages to cache it would be worth setting up a proxy. In your case
you're generating a lot of images, so a proxy might be worth it. But
if you're just caching your sitemap and all-records page, a proxy
would be overkill.
Besides the cache decorator there's a 'cache' object, which can cache
anything, not just pages. That's something a proxy can't do.
> however some of these images should only be seen by the user who owns them.
> so one solution would be to include a hash in the url.
Ideally you'd have an auth umbrella above both the proxy and the
Pylons application, but that means you have to use something other
than Python for your auth. Perhaps an Apache module.
By the way, the pylons-discuss list is the best place to ask usage
questions. The pylons-devel list is for discussing changes to Pylons;
details the entire userbase doesn't want to be bothered with.
--
Mike Orr <slugg...@gmail.com>
tor...@gmx.de
>
> On Tue, Oct 20, 2009 at 8:16 AM, <tor...@gmx.de> wrote:
> >
> > my app is dynamically rendering images using cairo, which i write into
> > the response with:
> >
> > response.content_type="image/png"
> > surface.write_to_png( response )
> >
> > i did not return anything, because it was working fine.
> > so when this was working, i wanted to have the images cached.
> >
> > @beaker_cache() did not work until i returned response.content
> > from my action.
>
> The decorator caches the return value, so that makes sense. It
> doesn't look inside the response to see what you've written to it.
i know that. i read the code.
consider it a suggestion to change the behaviour.
But considering a reimplementaion is scheduled. i dont come up with a
patch.
>
> > i also noticed that etag_cache() does not work when @beaker_cache is in
> > place. (clearly the function is never executed)
> >
> > it seems to be necessary to make etag_cache() a decorator too, where the
> > key may be a callable.
>
> I never could get etag_cache to work.
it works. if you dont use @beaker_cache
>
> > on irc it was pointed out, that most of this caching is not pylons
> > business. and i should be using a caching reverse proxy, and be all set.
>
> > what is the point in this caching infrastructure then anyways ?
>
> That's a rather one-size-fits-all statement. If you have a lot of
> pages to cache it would be worth setting up a proxy. In your case
> you're generating a lot of images, so a proxy might be worth it. But
> if you're just caching your sitemap and all-records page, a proxy
> would be overkill.
well... the generated images are generated for one particular user, so
basically i can just go with etag_cache()
>
> Besides the cache decorator there's a 'cache' object, which can cache
> anything, not just pages. That's something a proxy can't do.
>
> > however some of these images should only be seen by the user who owns them.
> > so one solution would be to include a hash in the url.
>
> Ideally you'd have an auth umbrella above both the proxy and the
> Pylons application, but that means you have to use something other
> than Python for your auth. Perhaps an Apache module.
>
> By the way, the pylons-discuss list is the best place to ask usage
> questions. The pylons-devel list is for discussing changes to Pylons;
> details the entire userbase doesn't want to be bothered with.
i dont want to ask usage questions.
i wrote this after i got it working.
i want to trigger a discussion about the caching layer.
i currently have some requirements which pylons does not fulfill, so i
need to roll some own stuff or patch pylons.
i prefer patching pylons, but i dont want to submit stuff before not
making sure the guy in charge likes the approach.
the etag_cache_decorator i need is easy:
-----------------------------------------------------------------------------
def etag_cache_decorator(key_callable=None, key_callable_with_args=None ):
def wrapper(func, *args, **kwargs):
if key_callable_with_args:
etag_cache( key_callable_with_args( *args, **kwargs ) )
elif key_callable:
etag_cache( key_callable() )
else:
etag_cache( "1" )
result = func(*args, **kwargs)
return result
return decorator(wrapper)
-----------------------------------------------------------------------------
maybe this is already enough.
if you still think this belongs to disuss then im gonna subscribe.
>
> --
> Mike Orr <slugg...@gmail.com>
>
--
torben Hohn
http://galan.sourceforge.net -- The graphical Audio language