Expire items in a list
Ericson Smith
We've been enthusiastic user of Redis here at funadvice.com. With millions of users monthly, we've been able to shift a lot out of Postgres and onto Redis.
We're now using it for user presence, with something simple as
GETSET "fap:username", 1
EXPIRE "fap:username", 300
And getting back user presence individually, or all, with:
KEYS "fap:*"
Which brings me to the point, is there any future way in the future that you're going to expire an item in a list? with the KEYS command, we're fetching all possibilities (which could be very high), but with a list we can nicely paginate through it.
I know we could handle a list with a cronjob, but Redis has spoiled us with GETSET and EXPIRE :-)
- Ericson Smith
CTO
http://www.funadvice.com
Salvatore Sanfilippo
> Hi!
>
> We've been enthusiastic user of Redis here at funadvice.com. With millions
> of users monthly, we've been able to shift a lot out of Postgres and onto
> Redis.
Cool! Can I use the name of your company in the "Who is using Redis"
page that I'm going to build? I noticed that we have a lot of
real-world users already but from the Redis site new users can get the
feeling this is a project everybody is testing but nobody is using for
real work.
> We're now using it for user presence, with something simple as
> GETSET "fap:username", 1
> EXPIRE "fap:username", 300
That's a good idea indeed. I guess every pageview you set the key
because you can't really tell or trap when the user leave the site. It
can be a good idea to add this into the EXPIRE manual page under the
"Design patterns" section. I'll do it in a moment.
> And getting back user presence individually, or all, with:
> KEYS "fap:*"
>
> Which brings me to the point, is there any future way in the future that
> you're going to expire an item in a list? with the KEYS command, we're
> fetching all possibilities (which could be very high), but with a list we
> can nicely paginate through it.
Indeed, that is an interesting problem. I think I've an idea about this.
Imagine to simply use a Redis Set for this stuff (you have a lot of
advantages, SORT with BY, LIMIT, that is, easy pagination and so on).
But.. there is no support for pagination in Set elements right?
There is a trick about this. Don't use a single set, but multiple ones.
Example: I see that the user "foobar" did a pageview. Cool! he is alive.
I'll SADD it in the following way:
SADD aliveusers_<time> foobar
But time is not actually the Unix time in seconds. It is instead:
time = current_unix_time - (current_unix_time % (60*5))
This means that our time will change every five minutes (make sure
that the web servers have the clocks more or less in sync).
So what happens?
that aliveusers_1241448600 will have the users online in at max in the
latest 5 minutes, aliveusers_1241448300 will be the previous one with
the users online in the past five minutes, and so on.
Now in order to know with a given approximation the users that where
online in the latest 5 - 10 minutes we have to perform the UNION of
the latest two sets. But we have SUNIONSTORE!
So we perform:
SUNIONSTORE aliveusers aliveusers_1241448600 aliveusers_1241448300
SORT aliveusers LIMIT 0 10 .... and so on
Make sure to set an EXPIRE (for example of 30 seconds) on the
"aliveusers" itself, so you don't need to recompute it every time, but
just when GET returns nil.
This is very cool and all but...w hat about the old aliveusers_<time>
keys? You can do a simple thing, from time to time perform a DELETE of
the latest keys no longer used, that is the current "time" subtracted
of 600, 900, 300*N...
My English sucks hard so please feel free to ask about every part of
may email that is not clear.
Cheers,
Salvatore
--
Salvatore 'antirez' Sanfilippo
http://invece.org
Ericson Smith
That sounds like it would do the job for us. The idea that I like is that we can keep hitting Redis on every page view of a logged in user, and Redis itself will automatically keep an updated list of users for us by way of EXPIRE.
The technique using SUNIONSTORE frankly I did not think about, that's pretty neat :-) All in all, this keeps us from having to manually "manage" the list!
You can certainly use us in the "Who is using Redis" page. We're using redis for:
* queues (lpop, rpush)
* user presence
* new stuff to read
* user activity stream (coming soon).
Thanks!
- Ericson