Re: Notification System using LPUSH

[Andy McCurdy]

I do pretty much exactly what you describe. However, I do all the LPUSH-ing in a backround async worker so that it doesn't slow down my app. Create as many workers as you need to process the notification work in the background.

-andy

On Sep 17, 2012, at 6:32 AM, Tobi Kremer <tobias...@gmail.com> wrote:

Hey guys,

like almost everybody else on the planet I'm trying to implement my own (not quite realtime) notification system for our high-traffic site. As you could have guessed, I opted for Redis as the storage. For every activity that happens, I basically do something like this:

SET activity:$id $json_data

To inform users, I decided to use LPUSH, like this:

LPUSH user:$user_id:notifications $activity_id

Unfortunately, when having to notify thousands of users, sending thousands of distinct LPUSH commands down the line is quite expensive (even with pipelining). So I'm wondering if a list is really my best bet for this use case and if so, if there is a possibility to push ONE value onto MULTIPLE lists with one command (so the exact opposite of what LPUSH already allows you, which is: push multiple values onto one list).

Thanks a lot!

P.S. As I understand it, PubSub is not a solution because I want the notifications to be available to users who are NOT on the platform when an activity happens.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To view this discussion on the web visit https://groups.google.com/d/msg/redis-db/-/9WoVcAJ_GikJ.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

[Josiah Carlson]

Tobi,

Background tasks are the solution 99% of the time. If you can; use them.

Depending on the semantics of your notification requirements, you may
be able to get by with one of two other solutions (that I've
conveniently already written). The first is basically a poll-based
chat server: https://gist.github.com/1045789 , and requires that all
clients know that a message will come on channel X, and everyone is
broadcasting on the same channel X (think of it like a poll-based
pubsub, where your choices of subscriptions are stated in every
query). Each client says "give me stuff on these channels...", and the
messages are persisted for specific duration. The alternative is a
version that is in section 6.5.2 of my book, which requires explicit
subscription/unsubscription from a given channel, but which otherwise
behaves exactly like pubsub without wildcards, is poll-based, and
messages are persisted until all intended recipients have received the
message.

My guess is that background tasks are what you are looking for, however.

- Josiah

[Tobi Kremer]

Hi Andy,

Am Montag, 17. September 2012 19:20:35 UTC+2 schrieb Andy McCurdy:

I do pretty much exactly what you describe. However, I do all the LPUSH-ing in a backround async worker so that it doesn't slow down my app. Create as many workers as you need to process the notification work in the background.

Thanks! May I ask what kind of background processing system you're using or is it a homegrown solution? 

I just made a quick comparison between directly executing the commands on the Redis store (Perl Redis client with pipelining) and just writing the commands to the local file system. With 2.000 commands this took 12ms for the file system approach and 370ms with Redis, so there's definitely hope for a local background worker to improve performance ... :)

Cheers,

Tobi

[Andy McCurdy]

We're a Python shop so we use Celery. If you want a Perl solution, I've heard good things about Gearman. There's also Resque if you want to venture into Ruby land.

One other thing to note: the web app (producer of the notification) doesn't necessarily need to know all the recipients. In our case, the web app writes out the activity with some meta data (timestamp, type of activity, who produced it, etc). The worker only receives the ID of the activity. It uses the ID to look it up and uses the meta data to figure out what users/objects are interested in being notified. This means the web app doesn't need to spend *any* time calculating or persisting the list of interested recipients. Save yourself another 12ms. :)

-andy

[Peter Scott]

In general, if you find yourself wanting a custom command, Lua scripts are something to consider. In this case, for example, you could use this script:

for i=1,#KEYS do

   redis.call('lpush', KEYS[i], ARGV[1])

end

To call this script, the command is something like

EVAL $script $number_of_lists $list1 $list2 ... $listn $data_to_lpush

This requires Redis 2.6, which is still "unstable", but I've been using it under heavy load in production for months now without a problem.

-Peter

[Tobias Kremer]

On 18.09.2012, at 19:43, Peter Scott wrote:

> In general, if you find yourself wanting a custom command, Lua scripts are something to consider. In this case, for example, you could use this script:

That's good to know, thanks :)

--Tobi

[Tobias Kremer]

On 18.09.2012, at 16:26, Andy McCurdy wrote:
> One other thing to note: the web app (producer of the notification) doesn't necessarily need to know all the recipients. In our case, the web app writes out the activity with some meta data (timestamp, type of activity, who produced it, etc). The worker only receives the ID of the activity. It uses the ID to look it up and uses the meta data to figure out what users/objects are interested in being notified. This means the web app doesn't need to spend *any* time calculating or persisting the list of interested recipients. Save yourself another 12ms. :)

That makes perfectly sense! Thanks for the idea! :)

--Tobi