Best practices for StackExchange.Redis sentinel failover?

[Kevin Burton]

I think I am almost to the point that I understand the Redis server with sentinels failover process but I am a little shaky on what is available the the StackExchange.Redis library to handle such a case. My initial attempt would be to have a ConnectionMultiplexer for the sentinels and for the master/save servers. I would start out by getting the master server from the master/slave ConnectionMultiplexer as well as an IDatabase instance and do the normal read/writes to the database. I would also subscribe to the "+odown" message from the centinels and when/if it is received then change the master/slave ConnectionMultiplexer so that the next time a database instance is requested the newly promoted master will be used. What am I missing? What facilities are there to make this "failover" easier and more reliable with the StackExchange library?

Thank you.

[Jamie Flint]

Hi Kevin,

I've been working with StackExchange.Redis and testing the failover support just today in fact. We have our ConnectionMultiplexer set up inside a Lazy<> object as this is being used on a website running inside IIS. From what I have seen, you do not need to worry about the ConnectionMultiplexer, when the master goes down and sentinel kicks in writes will start hitting the new master when it is available provided your SETs have the DemandMaster flag set. As our solution is for Redis to be a cache we are not worried about loosing some SETs when the master goes down, but it shouldn't be too difficult to queue up the SETs until the master becomes available. When the old master comes back and sentinel swaps it to be a slave the ConnectionMultiplexer will see this and update accordingly. 

Maybe you are over thinking this a little too much? 

[Marc Gravell]

Hi; I have to apologise: my direct support for sentinel in SE.Redis is... minimal. Largely because I'm *not really a sentinel user* (we're limping along with something hone-baked until redis-cluster is a viable option for us, and that includes failover etc internally), so I don't feel massive familiar / comfortable - and partly because of time / resource constraints. I did receive a PR the other day that I need to look through, although my initial thoughts is "it would be nice if this was moved further in to be automatic".

The only support SE.Redis offers at the moment for failover is related to our home-baked "sentinel lite", which largely involves some pub/sub to trigger rediscovery. The options for that are exposed via MakeMaster.

Marc (SE.Redis author)

On 16 Feb 2015 15:15, "Kevin Burton" <rkevin...@charter.net> wrote:

I think I am almost to the point that I understand the Redis server with sentinels failover process but I am a little shaky on what is available the the StackExchange.Redis library to handle such a case. My initial attempt would be to have a ConnectionMultiplexer for the sentinels and for the master/save servers. I would start out by getting the master server from the master/slave ConnectionMultiplexer as well as an IDatabase instance and do the normal read/writes to the database. I would also subscribe to the "+odown" message from the centinels and when/if it is received then change the master/slave ConnectionMultiplexer so that the next time a database instance is requested the newly promoted master will be used. What am I missing? What facilities are there to make this "failover" easier and more reliable with the StackExchange library?

Thank you.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Kevin Burton]

Thank you. This is most helpful. 

I guess I am not sure about how to interact with the ConnectionMutiplexer. Right now I have three sentinels. So to start with I have a ConnectionMutiplexer with the end points of the three sentinels and the ServiceName the name I have given to the master. So before anything goes down I can call the SentinelMaster method on the ConnectionMultiplexer instance and get the address and port of the master but I am not sure how that is turned into an IDatabase instance that does the actual Redis requests or an IServer instance to get statistics on the master without instantiating another ConnectionMutiplexer. I like you am using Redis as a cache so I am not concerned with writes using a IDatabase instance while the master is failing over. I am mainly concerned with using the right address and port of the new master when a new client is starting up. Like I said if the accepted procedure is to call SentinelMaster on the ConnectionMultiplexer instance and new up a new "master" ConnectionMultiplexer instance then so be it. Would you mind sharing the arguments you use to your ConnectionMultiplexer and some of the key Redis configuration settings (like DemandMaster)?

Thanks again.

--
You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/sf0oIO-CIa4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+u...@googlegroups.com.

[Kevin Burton]

Even explaining the "sentinel lite" would be helpful. I saw MakeMaster but I am not clear on its use. From my limited understanding I would 

1 create an instance of ConnectionMultiplexer with the end points of my sentinels.

2 subscribe to the +switch-master channel. When I get a notification I call MakeMaster method on the ConnectioMultiplexer instance created in 1.

Right? It seems that the call to SentinelMaster already returns the address and port of the newly promoted master. I guess I am not sure the purpose of MakeMaster?

Thanks again.

You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/sf0oIO-CIa4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+u...@googlegroups.com.