[ANN] Presence System with Redis

[Cihangir SAVAŞ]

Hey guys, 

I was working on a presence system project, wanted to share the result with you. Here is the blog post about the project and the github repo.

It supports Online/Offline/Status/StatusChanges functionality. Written in Go. Uses Redis as backend storage.

If anyone is interested in trying it out and/or reviewing the architecture, I would welcome any feedback!

Cheers!

---

Cihangir SAVAS

https://github.com/cihangir

https://twitter.com/cihangirsavas

[Josiah Carlson]

A quick read of your blog post suggested that you built your system in a certain way, and reading your Go source confirmed.

Your system notifies all people on the system of all online/offline activities of all users. Generally speaking, this is not how it is usually done, simply because not everyone needs (or wants) to know the online status of all users (it's also not practical to scale). The simplest solution, which is also the right solution, is for either the clients to report which users they care about (their friends), or for the server to keep a record of this information. Either way, with this list, you can do one of two things:

1. Change your pubsub from subscribing to command operations ("__keyevent@%d__:expire" and "__keyevent@%d__:set") to subscribing to key notifications ("__keyspace@%d__:<user id>") for all of a user's friends (each subscriber is actually subscribed to all of the users that a user is friends with and wants to see status updates for)

2. Keep one subscriber per Go process (subscribing to "__keyevent@%d__:del" and "__keyevent@%d__:set"), then keep a lookup table mapping from user ids to connections that want information about that user (Go-side pubsub), and have Go pick the clients that should get notifications for changes.

#1 is the simplest to do with your existing setup, and is scaled by adding more Redis server slaves (publish to the master) or switching to Redis cluster (Redis cluster propagates all publish calls to all cluster servers). #2 is easier to scale with a single Redis server (each Go server should be able to handle tens of thousands of clients with 100-200 friends each), and in practice minimizes total data transfer overhead for Redis to Redis, and Redis to Go server messaging.

I describe a variant of #2 as part of an API to offer something like the Twitter streaming API in chapter 8 of Redis in Action, though I am not sure that I describe the "single subscriber per process" optimization (I thought about building it for the book, but less code in book form = easier to read). In the book it is about filtering based on a variety of criteria, but for user status notification, there is a single criteria, and the method for routing notifications can be a lot simpler.

 - Josiah

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Thomas Love]

Can you justify using keyspace notifications like this, given the lack of reliability guarantee? 

Thomas

[Cihangir SAVAŞ]

Hi Josiah

Thanks for the review,

Event listening system is built for worker(s) that distribute status change messages,  so there will/can be multiple workers that listen/subscribe to those messages and they will implement the business logic. It is not meant for all clients will listen for all messages, as you mentioned it is impossible to scale. What i mean by businees logic is, may be we dont care about the friendship status, may be we dont have any friend at all, or you have a feature that users can set them as offline, while they are online, those are not part of presence system, completely your features.

For your first suggestion, i wanted to keep the logic simple, and didnt want to load all the hard part onto shoulders of redis, we can easily spin up more workers for message dispatching. Because if we create a new listener for every user, redis will have to process (user count) * (churn rate), with my way it is  (1 * churn rate)  messages.

For the second one, this is how you can implement it on top of my package right now, in fact with this way it is easier to scale with client side sharding, - connect to multiple redis instances, listen their events, use the same logic to distribute the events - i am currently implementing this one, will update again when it is ready.  https://cloudup.com/ckK2RQ1eoMw

Thanks again.

---

Cihangir SAVAS

https://github.com/cihangir

https://twitter.com/cihangirsavas

[Cihangir SAVAŞ]

Hi Thomas,

I had the same thoughts while developing this package, but when i divide it the into parts, it is acceptable(i didnt use reliable because, that word is used differently by some others) by my use cases, let me explain;

If redis is down, no message, big problems

If you say, pub/sub is dropping messages, i have never had problems like that with Redis

If you have unreliable network between redis and your application, this is what TCP over IP, you will have reliable transmission at the end,

If your application/worker is down, possible, you dont have any client to process/notify, no need to get messages from Redis,  because even you wont be able send Online/Offline requests to Redis, your clients should re-connect to other running instances of your app/worker.

One thing that can be problematic is, if your app/worker crashes while processing the message. I can also provide solutions to that but too much complexity.

Key point is here, your application/worker should not accept connections(for Online/Offline/Status/StatusChanges requests) if it is not able to connect to redis, and i think it is acceptable.

I will be happy to hear your/other's thoughts too  

[Thomas Love]

Using keyspace notifications as the sole means for maintaining system state is like using receipts as your sole means for knowing your bank balance. If you miss a single one you'll be wrong forever after. 

I haven't looked carefully at the implementation but it doesn't look like you have any way for clients to initialize state. Do they start up assuming nobody is online? What do they do if they lose connectivity to Redis? Do they assume everyone dropped? Otherwise how do they deal with leaking "online" objects that they did not receive matching "offline"s for? 

Thomas

--
You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/H99FWJ0jvpc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+u...@googlegroups.com.

[Cihangir SAVAŞ]

I see the correlation between your analogy and the keyspace notifications as -1. 

Please take a look at the the API 

It is so easy to get the current status of wanted keys, setting some keys as online/offline.

This package provides the primitives, rest is your business logic and your implementation details.

---

Cihangir SAVAS

https://github.com/cihangir

https://twitter.com/cihangirsavas