Feature request: ttl method for cache

[Piotr Gosławski]

Hi!

Since the contribution workflow is a bit confusing to me, I'll just leave it here.
I think django's cache needs ttl (time to live) method that would return
time left until specified key expires. I, for instance, needed one when writing
a rate limiter. Without it I would have to store twice as much key-value pairs
in cache. I ended up with a workaround that dynamically binds ttl method,
but only for locmem and redis, since those are the backends I use.

Here is my workoround, maybe it will be of some use:
https://github.com/p-tr0/snipets/blob/master/django_cache_ttl.py

I guess the proper way would be to add a method that just raises
NotImplementedError and then cover it one backend at a time.

[Russell Keith-Magee]

Hi Piotr,

Sounds like a reasonable proposal to me. Feel free to open a ticket, and maybe try your hand at writing a patch!

As far as the NotImplemented bit goes - I'd rather see this implemented for all officially supported backends, rather than only one backend. The only exception to this would be if there is a fundamental technical reason why TTL can't be implemented (or adequately faked).

Yours

Russ Magee %-)

[Sean Bleier]

Hi Piotr,

For what it's worth, I have an implementation of ttl on django-redis-cache:

https://github.com/sebleier/django-redis-cache/commit/626a3263c428cf59b1428f5fc2aa638efd77346a

It's slightly different from your snippet in that if the key does not exist (because it is expired or didn't exist in the first place), the returned value will be 0 instead of None.  None is reserved for non-volatile keys with no expirations.

–Sean

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/07aa224a-3716-48c0-8af6-757d875c6d88%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Florian Apolloner]

Hi Russ,

On Tuesday, May 6, 2014 2:54:52 AM UTC+2, Russell Keith-Magee wrote:

As far as the NotImplemented bit goes - I'd rather see this implemented for all officially supported backends, rather than only one backend. The only exception to this would be if there is a fundamental technical reason why TTL can't be implemented (or adequately faked).

Memcached doesn't provide access to the remaining TTL, and I don't see how we can reasonably fake this without writing an extra key containing the expiration date.

Cheers,
Florian

[Russell Keith-Magee]

/me facepalms

I realise now that I when I took a cursory look at the sample code that was provided, I read "LocMemCache" as "MemCache", and assumed that there was an exposed API that could be used to calculate TTL.

If this isn't the case, then I'll retract my support for this feature. If TTL isn't something memcache supports, then I'm not sure I see the value in adding this as a feature. The cache API really exists to provide a cache mockup when you don't have easy access to memcache (e.g., during development and testing); I'm not sure I see the benefit in adding support for a feature that isn't' supported by the most important implementation.

Russ %-)

[Piotr Gosławski]

W dniu wtorek, 6 maja 2014 11:47:17 UTC+2 użytkownik Florian Apolloner napisał:

[...]

Memcached doesn't provide access to the remaining TTL, and I don't see how we can reasonably fake this without writing an extra key containing the expiration date.

 Would that be unacceptable to add a switch in backend settings and also as extra argument for set() that would cause django to transparently save ttl as an extra key? It would obviously be disabled by default and docs would say about the extra memory this feature needs.

[Michael Manfre]

The majority of projects wouldn't use this and it's possible to accomplish with the existing cache API, so I don't think it makes sense to add this to Django. This functionality sounds like it is a better fit for a 3rd party app that provides custom cache backends. 

Regards,

Michael Manfre

[Thomas K. Adamcik]

See https://groups.google.com/d/msg/django-developers/ctKJzBTONu8/opbWqUIcOKgJ
for a similar case that has come up before. The tuple solution used there
could easily be adapted to store the time the key will expire, giving you the
information you need.

-Thomas

>
> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/913011d8-f4c4-4e7c-ac8f-6aa85869e669%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
mvh
Thomas Kongevold Adamcik

[Piotr Gosławski]

It would sacrifice atomicity of incr()/decr() methods and hit their speed pretty hard.

[Piotr Gosławski]

I was fixing my little helper function to behave more like Sean's and I think I've found a bug in locmem. Could you please take a look at this:

>>> cache
<django.core.cache.backends.locmem.LocMemCache object at 0x0000000004B077B8>
>>> cache.set('a', 1)
>>> cache.set('b', 2, None)
>>> cache.get('a')
1
>>> cache.get('b')
2
>>> cache.has_key('a')
True
>>> cache.has_key('b')
False

Either I'm missing something or has_key() is not working correctly for keys without expiration time.

[Malcolm Box]

On Thursday, 8 May 2014 16:59:55 UTC+1, Piotr Gosławski wrote:

I was fixing my little helper function to behave more like Sean's and I think I've found a bug in locmem. Could you please take a look at this:

Either I'm missing something or has_key() is not working correctly for keys without expiration time.

I think you're right - this is related to this bug: https://code.djangoproject.com/ticket/22495 and caused by the same patch.

File a ticket, and I'll have a look at fixing.

Malcolm

[Malcolm Box]

Ticket filed and pull request for fix https://code.djangoproject.com/ticket/22606

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/nwZBw64cD4c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/956bfca6-3ef7-4b4c-ac91-970628ec7d4b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Malcolm Box

CTO & Co-Founder, Tellybug  http://tellybug.com

mal...@tellybug.com   +44 7766 990123    @malcolmbox