In Redis Cluster the slave was not promoted to master when the Redis Instance(master) was stopped manually.

424 views
Skip to first unread message

Akshay Salunke

unread,
Aug 10, 2017, 3:11:02 AM8/10/17
to Redis DB
I was trying to check fail over of master node by stopping the master Redis node.
After stopping the master node, the slave was not promoted as master.

Below are the cluster nodes result.

asalu@redisdemo-vm:~$ redis-cli -c -h xx.xxx.xx.xx -p 7001 cluster nodes
3cf6cc233db19b99ba5bf04d6b680f3f7751b0e8 10.0.0.8:7001 myself,master - 0 0 2 connected 5461-10922
04c57200d7839cc3861b07eecafc3dee292094b6 xx.xxx.xx.xx:7000 slave 4d2d4777e370c8bb0b01ff74b9d333956f31d238 0 1502348407003 7 connected
d3430a54e3bf0ba01ed81e716a2ca169baeab08e xx.xxx.xx.xx:7004 slave 3cf6cc233db19b99ba5bf04d6b680f3f7751b0e8 0 1502348407504 5 connected
45a05d16d9e0a1ae29dc88abeb8eb2b347ef4d40 xx.xxx.xx.xx:7005 slave 68529ae6408ec9d93c351468883e3e7efae2506f 0 1502348408506 6 connected
4d2d4777e370c8bb0b01ff74b9d333956f31d238 :0 master,noaddr - 1501687236681 1501687235281 7 disconnected 0-5460
68529ae6408ec9d93c351468883e3e7efae2506f xx.xxx.xx.xx:7002 master - 0 1502348407504 3 connected 10923-16383
asalu@redisdemo-vm:~$


Can someone please help me understand why the slave was not promoted as master node?

Thanks in advance!!!!!!!

Jan-Erik Rediger

unread,
Aug 10, 2017, 4:37:47 AM8/10/17
to redi...@googlegroups.com
Check the log on all nodes. It should have more info.
> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.

Akshay Salunke

unread,
Aug 10, 2017, 7:37:38 AM8/10/17
to Redis DB, jan...@fnordig.de
Below are the logs for the restarted Redis Node instance (Also have one more Redis instance running on same machine): 

1:M 02 Aug 15:18:15.932 * No cluster configuration found, I'm 6cd2d79942ae46062a5ece3a70368c842595a541
.
.
.
.
1:M 02 Aug 15:18:15.967 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 02 Aug 15:18:15.967 # Server started, Redis version 3.2.9
1:M 02 Aug 15:18:15.967 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 02 Aug 15:18:15.967 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 02 Aug 15:18:15.967 * The server is now ready to accept connections on port 7003
1:M 02 Aug 15:20:36.707 # IP address for this node updated to 10.0.0.9
1:M 02 Aug 15:20:37.223 * Slave xx.xxx.xx.xx:7000 asks for synchronization
1:M 02 Aug 15:20:37.223 * Partial resynchronization not accepted: Runid mismatch (Client asked for runid '4e72d11fb74f241e8e4d3438c90748adb407c028', my runid is 'd0d88ce9ead1a3f13deb77bb1825da69b620d6d7')
1:M 02 Aug 15:20:37.223 * Starting BGSAVE for SYNC with target: disk
1:M 02 Aug 15:20:37.224 * Background saving started by pid 12
12:C 02 Aug 15:20:37.258 * DB saved on disk
12:C 02 Aug 15:20:37.258 * RDB: 6 MB of memory used by copy-on-write
1:M 02 Aug 15:20:37.348 * Background saving terminated with success
1:M 02 Aug 15:20:37.348 * Synchronization with slave xx:xxx:xx:xx:7000 succeeded


Please let me know if you get any information from these logs.

Jan-Erik Rediger

unread,
Aug 10, 2017, 10:07:47 AM8/10/17
to Akshay Salunke, Redis DB
To identify why a failover did not happen, the logs of the nodes taking
part in the voting are required.
The logs of a restarted node won't help.
> > an email to redis-db+u...@googlegroups.com <javascript:>.
> > > To post to this group, send email to redi...@googlegroups.com
> > <javascript:>.
Reply all
Reply to author
Forward
0 new messages