Redis Cluster: Switch a node from slave to master

3,523 views
Skip to first unread message

drecute

unread,
Sep 26, 2016, 7:40:26 AM9/26/16
to Redis DB
Hi,

I'm running a conventional 3 masters 3 slaves setup. While reconfiguring, I ended up have 4 slaves and 2 masters.

How do I switch a culprit slave to master?

Appreciate your help.

Tuco

unread,
Sep 28, 2016, 12:03:40 AM9/28/16
to Redis DB
You need to execute "cluster failover" command on the slave node to make it a master.

drecute

unread,
Sep 28, 2016, 5:32:57 AM9/28/16
to Redis DB
Thanks Tuco.

Hmm. It seems to work bit part

Here's what I did:

redis-trib.rb add-node 192.168.10.4:6379 192.168.10.98:6379 // 10.98 is master

The result of the above command is that 10.4 is added as a slave even though I didn't specify --slave flag

Now I ran cluster failover on the 10.4 node, but it makes no difference. Here's the result of cluster nodes

80df5261d2a8f40d81f3605a5a2e2853d68a309a 192.168.10.9:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475055015792 14 connected
a5a79bf0696bf279659f1a8cf79e0572ae135c65 192.168.10.98:6379 master - 0 1475055017296 14 connected 0-16383
5afa2b930bd38e4b474efd2e2173d35f012c6be1 192.168.10.6:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475055015290 14 connected
096d5917ead9b06297bd706ec6307b0a8676b4c2 192.168.10.4:6379 myself,slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 0 13 connected
08569accf3282b4ceb6b2726632b412418e0b728 192.168.10.8:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475055016794 14 connected

I need 10.4, 10.5, 10.98 as masters. Only 10.98 is a master right now.

I appreciate your help

Rahul Babbar

unread,
Sep 28, 2016, 9:51:17 AM9/28/16
to redi...@googlegroups.com

I think you need to do the following

1) do "cluster forget" on 10.4 and 10.5 so that they are not part of the same cluster.
2) connect to 10.98 and do "cluster meet 10.4" and then "cluster meet 10.5". Then 10.4 and 10.5 will be masters with no slots l.
3) "redis-trib.rb reshard 10.98" twice and then transfer some slots each to "10.4" and "10.5"

Then 10.4 and 10.5 will be masters with slots.

--
You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/jMeobwzWlhI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

drecute

unread,
Sep 28, 2016, 10:36:02 AM9/28/16
to Redis DB
Thanks Tuco.

But I think something is fundamentally. cluster meet 10.4 and cluster meet 10.5 did not make then masters after executing cluster forget [node id]

192.168.10.98:6379> cluster meet 192.168.10.4 6379
OK
192.168.10.98:6379> cluster meet 192.168.10.5 6379
OK
192.168.10.98:6379> cluster nodes
a5a79bf0696bf279659f1a8cf79e0572ae135c65 192.168.10.98:6379 myself,master - 0 0 14 connected 0-16383
5afa2b930bd38e4b474efd2e2173d35f012c6be1 192.168.10.6:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475073066985 14 connected
80df5261d2a8f40d81f3605a5a2e2853d68a309a 192.168.10.9:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475073064980 14 connected
e9b9f54e2df4b7459c50314195de7d8caea5d96a 192.168.10.5:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475073065281 14 connected
096d5917ead9b06297bd706ec6307b0a8676b4c2 192.168.10.4:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475073065582 14 connected
08569accf3282b4ceb6b2726632b412418e0b728 192.168.10.8:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475073065983 14 connected
To unsubscribe from this group and all its topics, send an email to redis-db+u...@googlegroups.com.

andyh

unread,
Sep 28, 2016, 10:56:21 AM9/28/16
to redi...@googlegroups.com
You should connect to the slave node and send `cluster meet {cluster_node}` command on that slave node instead of executing the command on the current master node. Hopefully that helps.

Of course, you might want to execute `cluster forget` first.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.



--
andyh
Andy Huang (Huangkejun)

drecute

unread,
Sep 28, 2016, 11:54:16 AM9/28/16
to Redis DB
Thanks Andy, but no difference.

[root@redis3 src]# ./redis-cli -h 192.168.10.5
192.168.10.5:6379> cluster meet 192.168.10.5 6379
OK
192.168.10.5:6379> cluster nodes
5afa2b930bd38e4b474efd2e2173d35f012c6be1 192.168.10.6:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475077884965 14 connected
80df5261d2a8f40d81f3605a5a2e2853d68a309a 192.168.10.9:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475077883461 14 connected
096d5917ead9b06297bd706ec6307b0a8676b4c2 192.168.10.4:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475077883461 14 connected
08569accf3282b4ceb6b2726632b412418e0b728 192.168.10.8:6379 slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 1475077883961 14 connected
a5a79bf0696bf279659f1a8cf79e0572ae135c65 192.168.10.98:6379 master - 0 1475077884463 14 connected 0-16383
e9b9f54e2df4b7459c50314195de7d8caea5d96a 192.168.10.5:6379 myself,slave a5a79bf0696bf279659f1a8cf79e0572ae135c65 0 0 14 connected

Perhaps I need to mention. I did run cluster forget [node id] for 10.4 & 10.5 only on 10.98
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.



--
andyh
Andy Huang (Huangkejun)

ma...@andyh.io

unread,
Sep 28, 2016, 12:02:42 PM9/28/16
to redi...@googlegroups.com
If this is the cluster u can do experiments with, another option is to delete the node.conf file because the cluster information is stored in that file.

Sent from my iPhone

drecute

unread,
Sep 28, 2016, 12:07:26 PM9/28/16
to Redis DB
Do you mean redis.conf file? Can't find node.conf file anywhere.

Rahul Babbar

unread,
Oct 2, 2016, 4:11:16 AM10/2/16
to redi...@googlegroups.com

Can you see whether the two nodes were still part of the cluster after you did a "cluster forget". I think they would be.. So you might have to force them to using " cluster failover force" I think. Which will give them a new node ID and they will be masters after meeting.

To unsubscribe from this group and all its topics, send an email to redis-db+unsubscribe@googlegroups.com.

drecute

unread,
Oct 13, 2016, 4:28:46 AM10/13/16
to Redis DB
Okay. This does not work.

Everytime I do `cluster failover` or `cluster failover force`, any master node becomes a slave, leaving only the current node as master
Reply all
Reply to author
Forward
0 new messages