REDIS HA with only 2 nodes

1,190 views
Skip to first unread message

Eric Fetzer

unread,
Aug 3, 2021, 7:41:59 PM8/3/21
to Redis DB
I'm brand new to REDIS, inherited an app that uses REDIS.  It's really sad, but I truly have only 2 nodes available per environment.  The customer is unwilling to drop the coin to add a third node.  At the same time, I've been given a requirement to provide outage free patching.  In researching HA in REDIS, I've read the documentation on REDIS Clustering as well as REDIS Sentinel.  It appears what my requirements dictate is unattainable based on documentation.  Is there a way in REDIS to configure this?  I'm not shooting for a load of 9's, I just want to be able to patch servers, deploy my application and such without outages.  Thanks!

Eric

Steve Lipinski

unread,
Aug 4, 2021, 9:16:35 AM8/4/21
to Redis DB
If you want automatic failover, yes - you need somewhere to add a third "something".  That does not have to be a full-blown Redis node, it could simply be somewhere to run a 3rd sentinel process (arbitrator).  This third "thing" could be an ultra small VM or server (hmm - wonder if you could run a 3rd sentinel on a Raspberry Pi :), I dunno, interesting thought... )
Essentially all the automatic failover mechanisms in Redis (Sentinel or Cluster) require quorum for decision making.  With only 2 servers, there is no ability for quorum when one is down.

Now, if you don't want automatic failover, you can use 2 nodes.  One is always master, one is replica.  When you perform upgrades, you upgrade the replica node first, make sure all is well with your upgraded application, perform a manual failover (gracefully stop redis master, convert replica to master, and set old-master as replica of new).  Then perform your upgrade of the second node (now a replica).  IT can be done fairly easily, just requiring manual management of the process.

Eric Fetzer

unread,
Aug 4, 2021, 9:54:10 AM8/4/21
to Redis DB
Thanks so much for your response Steve!  Hmmm, I like your concept of a "third something".  So that would be used simply to verify if either of the 2 nodes were down?  So how would I accomplish that?  Can you give me an example of what a "third something" might be at minimal?  We're running RHEL 7.9 VDI's.  Again, thanks!

Steve Lipinski

unread,
Aug 5, 2021, 8:56:15 AM8/5/21
to Redis DB
When using Redis Sentinel, you need a minimum of 3 Sentinels.  And, sentinel can be co-located with the Redis server processes.  So, given you have two servers, you can co-locate a sentinel with each, giving you 2 sentinels.  To get a third for quorum purposes, you can run a sentinel-only VM.  This one can be very small as the only purpose is for the sentinel process to run and provide that deciding vote.  There obviously needs to be network connectivity among these now-3 machines.
It kinda depends on the infrastructure that you're running on.  If on AWS, a micro EC2 instance should suffice.


Eric Fetzer

unread,
Aug 5, 2021, 4:07:33 PM8/5/21
to redi...@googlegroups.com
Thanks Steve!  Think I've found existing infrastructure that we can multi-task to add as a sentinel...

--
You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/Xhn5Th7y_n0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/3e1152b9-b3ef-44c4-a135-c2a81cb103ban%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages