Redis Multi-Master Replication among multiple DCs

[Will Hughes]

We've got two data centres which are geographically separated, each has it's own Redis infrastructure. 

For performance reasons, we don't want clients to talk to the other DC, and for reliability reasons, we can't guarantee connectivity between them. 

That said, we would like when network conditions permit it - for our Redis DBs to be synchronised between the DCs. Losing keys during a network partition is fine, as is having duplicate keys. In terms of conflict resolution/race conditions - whoever set first wins. 

We're using Redis as a cache, and we get the most benefit from it when the DBs are synced (hit rates go way up). 

In the end, I wrote a client that connects to both masters, subscribes to keyspace events, and then pushes all the SET commands between the DBs (all our keys are set with expiry, so I don't need to care about delete) - with an internal 'recently seen' cache to prevent replay loops. 

At the moment it's working fine, but with a couple of downsides. After we get a new key event, we then have to issue a GETEX to get the key and expiry  (doubling latency and network traffic), and during partitions any new keys are completely missed.  

None of these are show-stoppers for us at the moment, but reducing the network traffic would be nice.

I started looking as SYNC/PSYNC commands with the idea of being able to just replay the SETs to the other DB, but I can't find any indication of whether there might be obligations a client might have in pretending to be a slave. 

Are there better ways of handling this kind of limited replication? 

There was a similar discussion here almost exactly a year ago[1], but seemed to be trying for a more complete solution (caring about conflict resolution/etc), it also didn't seem to go anywhere. 

The rest of the discussions on this tend to try and wave people off from doing it - pushing towards either something client side (reading/writing to multiple redis servers), or Redis Cluster. 

- Will. 

[1] 'Redis multi master proof of concept (instead of cluster) feedback?' by Adam Liverman, 2014-09-28  - https://groups.google.com/forum/#!searchin/redis-db/replication/redis-db/jjosPGoKVVw/1hieUpD6l6UJ 

[Josiah Carlson]

Some of the details of replication is documented: http://redis.io/topics/replication The rest of the details are available in the Redis source code. I know a few of the requirements, but it's better to read the docs and source, because I will miss some of them.

 - Josiah

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.