Newbie: How to debug when redis cluster is in fail state

[sanjay ahuja]

Hi,

I am trying to learn redis and I had setup redis cluster few days back. It was working fine since yesterday. Now the cluster is being reported in fail state by "cluster info" command

Here is the output

127.0.0.1:6379> CLUSTER INFO

cluster_state:fail

cluster_slots_assigned:16384

cluster_slots_ok:12288

cluster_slots_pfail:0

cluster_slots_fail:4096

cluster_known_nodes:10

cluster_size:4

cluster_current_epoch:525

cluster_my_epoch:520

cluster_stats_messages_sent:14012420

cluster_stats_messages_received:13920791

How to debug and fix this issue? Are there any recommended steps.

Thanks in advance for any help here.

[Tuco]

you should try to execute "cluster nodes" and see which nodes are failing, and why?

[hokie...@gmail.com]

Hi Sanjay,

This means you are missing 1..n master servers that encompasses 4096 slots is/are down and no slave server(s) successfully restarted to get the slots back on-line. Until all of the slots are on master servers that are connected, your cluster will remain in a failed state.

As Tuco said, run cluster nodes to see which nodes failed and then go to the machines/Mesos to figure out what went wrong.

--John