No way currently... but actually I'm tempted of putting an option in 2.2 to issue a PUBLISH against some channel named like __keyspace__::expires for every expired key.
I know a few guys that could be very happy with this simple change.
On Thu, Sep 16, 2010 at 9:41 PM, Amit Rathore <amitrath...@gmail.com> wrote: > Hi folks,
> Is there any way to get notified when a key expires in Redis? If not > directly, then are there any common practices to approaching this?
> Regards, > Amit Rathore, > Runa, Inc.
> -- > You received this message because you are subscribed to the Google Groups "Redis DB" group. > To post to this group, send email to redis-db@googlegroups.com. > To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
> No way currently... but actually I'm tempted of putting an option in > 2.2 to issue a PUBLISH against some channel named like > __keyspace__::expires for every expired key.
> I know a few guys that could be very happy with this simple change.
> Cheers, > Salvatore
> On Thu, Sep 16, 2010 at 9:41 PM, Amit Rathore <amitrath...@gmail.com> > wrote: > > Hi folks,
> > Is there any way to get notified when a key expires in Redis? If not > > directly, then are there any common practices to approaching this?
> > Regards, > > Amit Rathore, > > Runa, Inc.
> > -- > > You received this message because you are subscribed to the Google Groups > "Redis DB" group. > > To post to this group, send email to redis-db@googlegroups.com. > > To unsubscribe from this group, send email to > redis-db+unsubscribe@googlegroups.com<redis-db%2Bunsubscribe@googlegroups.c om> > . > > For more options, visit this group at > http://groups.google.com/group/redis-db?hl=en.
> "We are what we repeatedly do. Excellence, therefore, is not an act, > but a habit." -- Aristotele
> -- > You received this message because you are subscribed to the Google Groups > "Redis DB" group. > To post to this group, send email to redis-db@googlegroups.com. > To unsubscribe from this group, send email to > redis-db+unsubscribe@googlegroups.com<redis-db%2Bunsubscribe@googlegroups.c om> > . > For more options, visit this group at > http://groups.google.com/group/redis-db?hl=en.
So we store what's equivalent to web-sessions in Redis, and they time-
out after 1 hour of inactivity. We accomplish this using the Redis key-
expiration mechanism. If we can get notified when a session expires,
we would like to run some reports. This will allow us to show real-
time metrics elsewhere.
Do you need more information?
Regards,
Amit.
On Sep 16, 3:36 pm, teleo <lev.ko...@gmail.com> wrote:
On Thu, Sep 16, 2010 at 9:40 PM, Salvatore Sanfilippo <anti...@gmail.com> wrote: > No way currently... but actually I'm tempted of putting an option in > 2.2 to issue a PUBLISH against some channel named like > __keyspace__::expires for every expired key.
> I know a few guys that could be very happy with this simple change.
Make that two guys now, and you have a set of free <insert favorite beverage here> waiting for you at Codebits 2010 :)
On Thu, Sep 16, 2010 at 11:59 PM, Jeremy Zawodny <Jer...@zawodny.com> wrote: > Yeah, it has come up several times over the last year. > Jeremy
Yeah... I try to resist to see if something is *really* needed in order to avoid bloating Redis, but it is possible that now is about time?
Now I need a collective effort to give me the power :) That is, please give me use cases that you guys are experimenting at work where this could be useful.
I'll try with a few:
1) You can use it as a timer to do expiring of other things. For instance you want to expire set elements, what you can do is having two keys for every unix time in the future where you have at least one "set element expire" active: one of the keys is just a dummy key with an expire set at the same time (see EXPIREAT). The other key is a list of set elements I want to expire when this happens. So when I get the notification, I read the list, and remove the set elements. It's wise to also set a (more far in time) expire in the list key as well, so that if you miss the notification, the cleanup of the two keys still happen.
2) You have workers updating your cache, that is composed of elements that should not be invalidated for five minutes. You want to rebuild the expired entries ASAP once they expire. So you can subscribe to the expires channel and put what you get in the stream into a list: other workers will BLPOP from this list and rebuild the cache.
3) There is some need to collect stats about the most used items in your cache, that are probably the most used items in your application. It's simply possible to subscribe to the expires and every time one entry expires we can increment a counter or likewise.
4) I need to track the number of users online. Every time I see an user I add the ID into a Set. Every time I see the session expiring I remove it from the Set.
Ok honestly this use cases are weak. I need something better to go forward :)
Umm, I think I should line up with the 'Usual Suspects' :)
What I am doing now doesn't really require this feature, but I still think that Redis need to be able to externalize this aspect of its behaviour, because from an operational view, you need to be able to track down what is going on your server.
On Thu, Sep 16, 2010 at 5:29 PM, Jeremy Zawodny <Jer...@zawodny.com> wrote: > Yeah, it has come up several times over the last year. > Jeremy
> On Thu, Sep 16, 2010 at 1:40 PM, Salvatore Sanfilippo <anti...@gmail.com> > wrote:
>> No way currently... but actually I'm tempted of putting an option in >> 2.2 to issue a PUBLISH against some channel named like >> __keyspace__::expires for every expired key.
>> I know a few guys that could be very happy with this simple change.
>> Cheers, >> Salvatore
>> On Thu, Sep 16, 2010 at 9:41 PM, Amit Rathore <amitrath...@gmail.com> >> wrote: >> > Hi folks,
>> > Is there any way to get notified when a key expires in Redis? If not >> > directly, then are there any common practices to approaching this?
>> > Regards, >> > Amit Rathore, >> > Runa, Inc.
>> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Redis DB" group. >> > To post to this group, send email to redis-db@googlegroups.com. >> > To unsubscribe from this group, send email to >> > redis-db+unsubscribe@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/redis-db?hl=en.
>> "We are what we repeatedly do. Excellence, therefore, is not an act, >> but a habit." -- Aristotele
>> -- >> You received this message because you are subscribed to the Google Groups >> "Redis DB" group. >> To post to this group, send email to redis-db@googlegroups.com. >> To unsubscribe from this group, send email to >> redis-db+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/redis-db?hl=en.
> -- > You received this message because you are subscribed to the Google Groups > "Redis DB" group. > To post to this group, send email to redis-db@googlegroups.com. > To unsubscribe from this group, send email to > redis-db+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/redis-db?hl=en.
> Umm, I think I should line up with the 'Usual Suspects' :)
> What I am doing now doesn't really require this feature, but I still > think that Redis need to be able to externalize this aspect of its > behaviour, because from an operational view, you need to be able to > track down what is going on your server.
Not convincing Anibal ;) Because stated in this way sounds like "for debugging purposes".
We need something more than this... otherwise we could add this in the DEBUG command or alike.
> On Thu, Sep 16, 2010 at 5:29 PM, Jeremy Zawodny <Jer...@zawodny.com> wrote: >> Yeah, it has come up several times over the last year. >> Jeremy
>> On Thu, Sep 16, 2010 at 1:40 PM, Salvatore Sanfilippo <anti...@gmail.com> >> wrote:
>>> No way currently... but actually I'm tempted of putting an option in >>> 2.2 to issue a PUBLISH against some channel named like >>> __keyspace__::expires for every expired key.
>>> I know a few guys that could be very happy with this simple change.
>>> Cheers, >>> Salvatore
>>> On Thu, Sep 16, 2010 at 9:41 PM, Amit Rathore <amitrath...@gmail.com> >>> wrote: >>> > Hi folks,
>>> > Is there any way to get notified when a key expires in Redis? If not >>> > directly, then are there any common practices to approaching this?
>>> > Regards, >>> > Amit Rathore, >>> > Runa, Inc.
>>> > -- >>> > You received this message because you are subscribed to the Google >>> > Groups "Redis DB" group. >>> > To post to this group, send email to redis-db@googlegroups.com. >>> > To unsubscribe from this group, send email to >>> > redis-db+unsubscribe@googlegroups.com. >>> > For more options, visit this group at >>> > http://groups.google.com/group/redis-db?hl=en.
>>> "We are what we repeatedly do. Excellence, therefore, is not an act, >>> but a habit." -- Aristotele
>>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Redis DB" group. >>> To post to this group, send email to redis-db@googlegroups.com. >>> To unsubscribe from this group, send email to >>> redis-db+unsubscribe@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/redis-db?hl=en.
>> -- >> You received this message because you are subscribed to the Google Groups >> "Redis DB" group. >> To post to this group, send email to redis-db@googlegroups.com. >> To unsubscribe from this group, send email to >> redis-db+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/redis-db?hl=en.
> -- > You received this message because you are subscribed to the Google Groups "Redis DB" group. > To post to this group, send email to redis-db@googlegroups.com. > To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
Choosing between DEBUG and EXPIRE subscription, DEBUG gets my money for sure ;)
The problem with EXPIRE subscription, in the end is that you can't be sure that the process handling it will be always there, so there is some opportunity for an inconsistent state to arise. So in the end a better solution, is having the appropiate structures for tracking what you want to get rid off, if it is really required.
But from a Data Structure Server point of view, nothing happens in Redis without knowing the result, _except_ expiration of KEYS, I have been faced with the duty of keeping a Redis instance, that was pushed far beyond its limits, up and running and from a operational point of view, I should be able to collect as much information as possible about what is going on in the server minimizing the impact of gathering it.
<anti...@gmail.com> wrote: > 2010/9/17 Aníbal Rojas <anibalro...@gmail.com>: >> Umm, I think I should line up with the 'Usual Suspects' :)
>> What I am doing now doesn't really require this feature, but I still >> think that Redis need to be able to externalize this aspect of its >> behaviour, because from an operational view, you need to be able to >> track down what is going on your server.
> Not convincing Anibal ;) Because stated in this way sounds like "for > debugging purposes".
> We need something more than this... otherwise we could add this in the > DEBUG command or alike.
>> On Thu, Sep 16, 2010 at 5:29 PM, Jeremy Zawodny <Jer...@zawodny.com> wrote: >>> Yeah, it has come up several times over the last year. >>> Jeremy
>>> On Thu, Sep 16, 2010 at 1:40 PM, Salvatore Sanfilippo <anti...@gmail.com> >>> wrote:
>>>> No way currently... but actually I'm tempted of putting an option in >>>> 2.2 to issue a PUBLISH against some channel named like >>>> __keyspace__::expires for every expired key.
>>>> I know a few guys that could be very happy with this simple change.
>>>> Cheers, >>>> Salvatore
>>>> On Thu, Sep 16, 2010 at 9:41 PM, Amit Rathore <amitrath...@gmail.com> >>>> wrote: >>>> > Hi folks,
>>>> > Is there any way to get notified when a key expires in Redis? If not >>>> > directly, then are there any common practices to approaching this?
>>>> > Regards, >>>> > Amit Rathore, >>>> > Runa, Inc.
>>>> > -- >>>> > You received this message because you are subscribed to the Google >>>> > Groups "Redis DB" group. >>>> > To post to this group, send email to redis-db@googlegroups.com. >>>> > To unsubscribe from this group, send email to >>>> > redis-db+unsubscribe@googlegroups.com. >>>> > For more options, visit this group at >>>> > http://groups.google.com/group/redis-db?hl=en.
>>>> "We are what we repeatedly do. Excellence, therefore, is not an act, >>>> but a habit." -- Aristotele
>>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Redis DB" group. >>>> To post to this group, send email to redis-db@googlegroups.com. >>>> To unsubscribe from this group, send email to >>>> redis-db+unsubscribe@googlegroups.com. >>>> For more options, visit this group at >>>> http://groups.google.com/group/redis-db?hl=en.
>>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Redis DB" group. >>> To post to this group, send email to redis-db@googlegroups.com. >>> To unsubscribe from this group, send email to >>> redis-db+unsubscribe@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/redis-db?hl=en.
>> -- >> You received this message because you are subscribed to the Google Groups "Redis DB" group. >> To post to this group, send email to redis-db@googlegroups.com. >> To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com. >> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
> "We are what we repeatedly do. Excellence, therefore, is not an act, > but a habit." -- Aristotele
> -- > You received this message because you are subscribed to the Google Groups "Redis DB" group. > To post to this group, send email to redis-db@googlegroups.com. > To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
Why do you say that use case #4 is weak? I have pretty much exactly
that use case on my project (I don't just count the number of users
online, but a bunch of other reporting).
How can I implement this without getting notified on key expiry?
I'm asking because if there's some way of doing this without polling
of some kind, I'd love to know.
Regards,
Amit.
On Sep 17, 1:34 am, Salvatore Sanfilippo <anti...@gmail.com> wrote:
> On Thu, Sep 16, 2010 at 11:59 PM, Jeremy Zawodny <Jer...@zawodny.com> wrote:
> > Yeah, it has come up several times over the last year.
> > Jeremy
> Yeah... I try to resist to see if something is *really* needed in
> order to avoid bloating Redis, but it is possible that now is about
> time?
> Now I need a collective effort to give me the power :) That is, please
> give me use cases that you guys are experimenting at work where this
> could be useful.
> I'll try with a few:
> 1) You can use it as a timer to do expiring of other things. For
> instance you want to expire set elements, what you can do is having
> two keys for every unix time in the future where you have at least one
> "set element expire" active: one of the keys is just a dummy key with
> an expire set at the same time (see EXPIREAT). The other key is a list
> of set elements I want to expire when this happens. So when I get the
> notification, I read the list, and remove the set elements. It's wise
> to also set a (more far in time) expire in the list key as well, so
> that if you miss the notification, the cleanup of the two keys still
> happen.
> 2) You have workers updating your cache, that is composed of elements
> that should not be invalidated for five minutes. You want to rebuild
> the expired entries ASAP once they expire. So you can subscribe to the
> expires channel and put what you get in the stream into a list: other
> workers will BLPOP from this list and rebuild the cache.
> 3) There is some need to collect stats about the most used items in
> your cache, that are probably the most used items in your application.
> It's simply possible to subscribe to the expires and every time one
> entry expires we can increment a counter or likewise.
> 4) I need to track the number of users online. Every time I see an
> user I add the ID into a Set. Every time I see the session expiring I
> remove it from the Set.
> Ok honestly this use cases are weak. I need something better to go forward :)
On Fri, Sep 17, 2010 at 8:48 PM, Amit Rathore <amitrath...@gmail.com> wrote: > Why do you say that use case #4 is weak? I have pretty much exactly > that use case on my project (I don't just count the number of users > online, but a bunch of other reporting). > How can I implement this without getting notified on key expiry?
> I'm asking because if there's some way of doing this without polling > of some kind, I'd love to know.
Hello Amit,
there is a very simple way to get this function just using the features we already have. Use a DB just for this (SELECT it). Every time you see an user perform a SETEX of user:<id>, with the timeout you like. To check if an user exists just do EXISTS user:<id>. To count the number of users just check with DBSIZE. Get the list of all the online users with "KEYS *".
> On Fri, Sep 17, 2010 at 8:48 PM, Amit Rathore <amitrath...@gmail.com> wrote:
> > Why do you say that use case #4 is weak? I have pretty much exactly
> > that use case on my project (I don't just count the number of users
> > online, but a bunch of other reporting).
> > How can I implement this without getting notified on key expiry?
> > I'm asking because if there's some way of doing this without polling
> > of some kind, I'd love to know.
> Hello Amit,
> there is a very simple way to get this function just using the
> features we already have.
> Use a DB just for this (SELECT it). Every time you see an user perform
> a SETEX of user:<id>, with the timeout you like.
> To check if an user exists just do EXISTS user:<id>. To count the
> number of users just check with DBSIZE. Get the list of all the online
> users with "KEYS *".
I joined this thread to share my point of view regarding this topic,
may be that tells how useful it could be in various real life problems
for me. Below are few that could be used and it's must have for us.
- Application start some task which is kind fire and forget, but it's
critical so application need to make sure it happens. To support that
application need to go back after X amount of time(5 min) to verify
that it really happened. This could be achieved with redis key expire
notification.
- Application need to fire scheduled task everyday at 5 am that could
be achieved with key expire. This is like using it scheduler but it's
added benefit who want to implement it.
Above can be achieved with other solutions but most don't scale well.
On Sep 16, 6:36 pm, teleo <lev.ko...@gmail.com> wrote:
On Thu, Oct 14, 2010 at 11:13 AM, mamu <manishtpa...@gmail.com> wrote: > I joined this thread to share my point of view regarding this topic, > may be that tells how useful it could be in various real life problems > for me. Below are few that could be used and it's must have for us.
Must-haves are much easier to accept when they include a patch ;)
> - Application start some task which is kind fire and forget, but it's > critical so application need to make sure it happens. To support that > application need to go back after X amount of time(5 min) to verify > that it really happened. This could be achieved with redis key expire > notification.
The application is going to check after 5 minutes, and you want to use Redis as a timer to signal that it should check? One thing to remember about Redis expiration is that it's not a hard 5 minutes. Keys that are supposed to expire in 5 minutes may never expire, or they could expire in one second (if there is memory pressure). You're better off using an explicit event queue with timing. RabbitMQ, ActiveMQ, and other Redis-based solutions all have the ability to say "execute command X in Y seconds".
> - Application need to fire scheduled task everyday at 5 am that could > be achieved with key expire. This is like using it scheduler but it's > added benefit who want to implement it.
Not really. Redis key expiration may occur before it's desired expiration, and it could also expire never. It's not a hard deadline, it's not a scheduler, and it's not cron.
> Above can be achieved with other solutions but most don't scale well.
Actually, many scale pretty well. For example, ActiveMQ will cluster if necessary, but it will also move tens of thousands of items through the system each second on modern hardware. If you just need a simple work queue, ZeroMQ claims to be able to move upwards of 2.8 million messages/second (for 8 byte messages), can do replication, tree structures, etc.
Heck, a Redis ZSET is arguably close to perfect for scheduled tasks, though you need to have one process that pulls tasks off of the scheduled list and puts it on a regular LIST for multiple readers. Multiple writers are trivial (put stuff in the list that you want to happen *now*, stuff in the ZSET that you want to happen in the future). You could easily do a few tens of thousands of events/second without breaking a sweat. And also *actually* get what you want (check in 5 minutes/check at 5AM, vs check some time between now and the future).
What scales are you talking about that "don't scale",
> On Sep 16, 6:36 pm, teleo <lev.ko...@gmail.com> wrote: >> Hi Amit, I am curious about the use case. How would you use the >> notifications, if they were supported?
>> On Sep 16, 9:41 pm, Amit Rathore <amitrath...@gmail.com> wrote:
>> > Hi folks,
>> > Is there any way to get notified when a key expires in Redis? If not >> > directly, then are there any common practices to approaching this?
>> > Regards, >> > Amit Rathore, >> > Runa, Inc.
> -- > You received this message because you are subscribed to the Google Groups "Redis DB" group. > To post to this group, send email to redis-db@googlegroups.com. > To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
