add CACHE_KEY_PREFIX functionality
Nowell Strite
I noticed that the caching framework does not currently support
setting a CACHE_KEY_PREFIX that would automatically be prefixed to any
get/set/delete function call in the caching. I would be willing to
write the patch, I just wanted to make sure that I was not missing a
discussion thread on this topic.
Django currently supoorts COOKIE prefixes, as well as a
CACHE_MIDDLEWARE_KEY_PREFIX, but I would find it immensely useful to
have a prefix for all cache key entries (i.e. site's sharing a common
caching mechanism, with overlapping keys that hold different data)
If this idea is appropriate, it shouldn't take more than an hour or so
to make a patch for all cache backends.
Thoughts?
Nowell
Jacob Kaplan-Moss
> Django currently supoorts COOKIE prefixes, as well as a
> CACHE_MIDDLEWARE_KEY_PREFIX, but I would find it immensely useful to
> have a prefix for all cache key entries (i.e. site's sharing a common
> caching mechanism, with overlapping keys that hold different data)
I'm not a big fan of having my cache keys transparently changed. I
think as a point of philosophy Django shouldn't mess with things like
that.
Besides, doing it yourself couldn't be easier::
cache.set(settings.CACHE_KEY_PREFIX + key, value, timeout)
Jacob
Nowell Strite
second.
Example:
cache.get(settings.CACHE_KEY_PREFIX + key)
cache.set(settings.CACHE_KEY_PREFIX + key, value, timeout)
cache.delete(settings.CACHE_KEY_PREFIX + key)
cache.has_key(settings.CACHE_KEY_PREFIX + key)
Since it would be a global prefix, it would be transparent to the
Django developer (if you want to assign your own prefix, then leave
the settings.CACHE_KEY_PREFIX blank and do it yourself--
cache.set(settings.MY_CACHE_KEY_PREFIX + key, value)... but since you
will always want to prefix your cache keys (in the case of a shared
caching mechanism) I think it would be beneficial to implement this--
we already do for set_cookie/get_cookie/delete_cookie, so why not
here?
I think the same logic/discussion that comes into play with the
set_cookie prefix applies here as well, don't you think?
Thanks!
Nowell
On Apr 3, 2:29 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:
Adrian Holovaty
> Django developer (if you want to assign your own prefix, then leave
> the settings.CACHE_KEY_PREFIX blank and do it yourself--
> cache.set(settings.MY_CACHE_KEY_PREFIX + key, value)... but since you
> will always want to prefix your cache keys (in the case of a shared
> caching mechanism) I think it would be beneficial to implement this--
> we already do for set_cookie/get_cookie/delete_cookie, so why not
> here?
I can see the value of a global CACHE_KEY_PREFIX setting, but we're
always hesitant to add new settings. What if we removed the
CACHE_MIDDLEWARE_KEY_PREFIX, in favor of a new CACHE_KEY_PREFIX
setting? Is there a case where the two settings would need to differ?
Adrian
--
Adrian Holovaty
holovaty.com | djangoproject.com
Nowell Strite
I can definitely understand the hesitancy to add new settings :-)
I do think that your proposal to broaden the scope of the existing
CACHE_MIDDLEWARE_KEY_PREFIX is a good idea (I cannot think of a case
where the two settings would need to differ). I was trying to avoid
proposing a backward incompatible change that you would need to
announce (if you guys are comfortable renaming and/or re purposing the
existing CACHE_MIDDLEWARE_KEY_PREFIX, than you have my vote for that
as well).
I will make an initial patch tomorrow, and post it to
djangoproject.com ticket system for more review/discussion.
On Apr 3, 5:19 pm, "Adrian Holovaty" <holov...@gmail.com> wrote:
Ian Holsman
It also doesn't work on multi-site installations.
It also stops developers having the ability to use the cache at the
global granularity or to
use cache keys from other applications sometimes.
regards
Ian
--
Ian Holsman
I...@Zilbo.com
http://personalinjuryfocus.com/
Adam Seering
Reading this thread from a while back: Has support for something
like CACHE_KEY_PREFIX been added? I assume not, based on this thread;
I can't find it, at least...
I'm writing a patch for my own use to do exactly this, because I need
it in order to deploy multiple Django-based apps on a single server.
I feel like this must be a common scenario.
I don't quite understand how this stops developer thinking; on the
contrary, I think developers should be asked to think before writing
apps that alter the caches of other programs. As a developer, when I
use Django's caching API, I expect that someone has abstracted all
these annoying namespacing issues away for me.
If people really need to get at the global cache namespace, support
could be added for, ie.,
cache.get(myKey, use_global_cache_namespace=True)
which would ignore CACHE_KEY_PREFIX. Between this and allowing
CACHE_KEY_PREFIX to be blank or unspecified, I feel like everyone
would get what they want, in a relatively elegant way.
Thanks,
Adam
Jacob Kaplan-Moss
This is how I see it.
You're asking to add magic behavior to cache.get/cache.set so that you can say::
cache.get("key")
instead of::
cache.get(settings.CACHE_PREFIX + "key")
I've learned from hard experience that hiding behavior just to save a
few characters of code simply isn't worth the negative effect on the
learning curve.
Jacob
Adam Seering
What I'm really asking for is for "cache" to refer to "My personal
cache, for use by my code", not "some generic memory pool that can get
corrupted by others arbitrarily". The adding of a text string is
essentially an implementation detail for me; I'm certainly open to
other alternatives.
"learning curve": Actually, I've found the current learning curve to
be a bit steep, and one of the advantages of this suggested patch
would be that it would become shallower. I would like it to be:
"If you put data in with put(), you can get it back with get()"
Without this feature, it (currently) is:
"If you put data in with put(), you can get it back with get(). Well,
unless some other developer writing code for a different app on your
system, uses a key with the same name. In this case, get() will
return something very different. Or, if you're running two very
similar versions of your code on the same computer, for debugging or
the like, you're going to get back subtly different cached objects,
and it's going to take you forever and a day to debug it. So you
don't want to do that."
So, I'd really like this feature, if only for ease-of-learning and
DRY-ness.
Adam
Jacob Kaplan-Moss
> I would like it to be:
>
> "If you put data in with put(), you can get it back with get()"
See, from where I sit, your proposal changes this to: "if you put data
in with set() [it's set, not put, FYI], you can get it back with get()
unless you changed your settings module or someone changed
CACHE_KEY_PREFIX."
With ``cache.set(settings.CACHE_KEY_PREFIX + my_key)`` you know
*exactly* what's happening; any sort of hidden behavior is, well,
hidden.
Can you at least see where I'm coming from here?
Is there something I'm missing here? Is it really all that onerous to
put a prefix in your code somewhere? Or to use something like
http://dpaste.com/hold/37283/?
Jacob
Adam Seering
On Feb 28, 2008, at 7:30 PM, Jacob Kaplan-Moss wrote:
>
> On 2/28/08, Adam Seering <asee...@gmail.com> wrote:
>> I would like it to be:
>>
>> "If you put data in with put(), you can get it back with get()"
>
> See, from where I sit, your proposal changes this to: "if you put data
> in with set() [it's set, not put, FYI]
Sorry; yeah, been reading too much random code lately...
> , you can get it back with get()
> unless you changed your settings module or someone changed
> CACHE_KEY_PREFIX."
Well, that's already equivalently true if you, say, swap the
DATABASE_* settings in settings.py (ie., you get the database for the
other app). So I think that's an expectable failure mode.
> With ``cache.set(settings.CACHE_KEY_PREFIX + my_key)`` you know
> *exactly* what's happening; any sort of hidden behavior is, well,
> hidden.
>
> Can you at least see where I'm coming from here?
I can see your point, and I agree that your method is much more
explicit.
However, I personally don't necessarily want explicit. For example,
if I create class MyTable(models.Model):, I don't always care exactly
what SQL is used to generate or access the table. I trust you to do
that. I just want my table.
If a lot of people do actually care about caching implementation, then
sure, your point makes sense.
> Is there something I'm missing here? Is it really all that onerous to
> put a prefix in your code somewhere? Or to use something like
> http://dpaste.com/hold/37283/?
Well, we use .get() and .set() about 80 times in our code currently,
and that will probably increase. So, it'd be a pain to maintain
custom code for each.
I could write my own helper class, to replace Django's cache object.
However, I do use Django's built-in caching helper functions as well,
so I would have to wrap all of them, too.
The practical reason that I want this code change is that, without it,
there's no good way for me to make use of a lot of Django's caching
functionality. Basically, in order to get my app to work on a shared
memcached install, I'd have to wrap or rewrite each Django cache
helper function. There are a lot of nice functions out there; I don't
really want to write special code for each of them...
EDIT: Ah, a helper class can be made that does roughly what I want. <http://www.djangosnippets.org/snippets/624/
>, for the logs. This still feels like a rough patch of Django for
me, but that code should suffice for now.
Thanks,
Adam
>
> Jacob
>
> >
Malcolm Tredinnick
On Thu, 2008-02-28 at 20:56 -0500, Adam Seering wrote:
[...]
> The practical reason that I want this code change is that, without it,
> there's no good way for me to make use of a lot of Django's caching
> functionality. Basically, in order to get my app to work on a shared
> memcached install, I'd have to wrap or rewrite each Django cache
> helper function. There are a lot of nice functions out there; I don't
> really want to write special code for each of them...
It's very easy to write your own caching backend that takes an existing
caching backend and extends it as you are asking here. This shouldn't
require any core changes at all. One of the advantages of modularity.
The only small downside is that we only look for cache backends in one
particular directory, but ... meh... it's not like that's an
insurmountable problem and a reasonable patch to make it behave similar
to database backend importing would at least be read seriously.
Malcolm
--
Depression is merely anger without enthusiasm.
http://www.pointy-stick.com/blog/