Reliable Pubsub Messaging

[Jon L]

Hi,

We currently use Redis for some storage of data, but I am considering
using pubsub to handle status messages between components...we are
transitioning to a real-time processing framework, but need to be able
to interface with this from outside of the system (ex. the webserver,
etc)...and I want to use pubsub to do this.

My main concern is regarding the safety of this...how do most people
in practice ensure that messages aren't missed, and dropped on the
floor? This is my main concern, that a message is missed, and a client
is stuck because it's not receiving the "done" response. Am I
approaching this problem right?



Thanks!

[Josiah Carlson]

You can't guarantee reliable pubsub messaging inside Redis. That's not what the feature is for. In fact, few pubsub systems offer that sort of reliability.

On the other hand, if you are willing to add a bit more, you might be able to get what you want. If each web client could be waiting on its own channel to receive its own "done" response, then what you're really looking for is point-to-point messaging, and/or you are looking for a concept in the task processing world called a "tombstone". The first one is easier to get, so I'll explain that one.

You want your client to be able to get the message when it's ready, and you don't want it to accidentally miss it. Solution: use Redis LISTs. You can have your web client BLPOP on a key that only it (and whatever is going to be sending "done") knows about (you can generate random keys for this, passing it to whatever will be sending "done"). Whatever would be sending the "done" message then RPUSHes the message to the key, while also setting the expiration time (say 30-60 minutes) to automatically clean up after itself if the web client never tries to fetch the message. That's it.

If this won't work for you, it would help if you might be able to give us more information about your specific problem, what you are trying to accomplish overall, etc.

Regards,

 - Josiah

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[Jon]

Josiah,

Thanks for your reply. I ended up doing essentially what you suggested, and it seems to be working great!

Thanks again