Cluster Failover For Persistent Timeouts?

[hh90]

We have a cluster of 6 nodes (3 masters, 3 slaves). An EC2 instance that hosts one of our master became unreachable, and one of the services using it kept logging: “redis: connection pool timeout”. The service couldn’t connect to the master node it was using, and wasn’t trying to reconnect to a working master node.

Is this expected behavior? Maybe this is something the Redis client needs to implement instead.

We were thinking of rescuing the “connection pool timeout” exception and trying to re-connect to a master that’s reachable if this happens again.

The Redis client we’re using is go-redis (github.com/go-redis/redis).

We’re using Redis Cluster 3.0.6.