Issue With Sentinel Montoring Redis Cluster With Instances Bound to Sub Network

[Ben McConnell]

Hello Redis Community!

I've recently started setting up a Redis cluster at my place of work and have had great success with it so far (outside of the problems I've had lately). I have successfully setup a HA cluster with one master and two slaves, with each node running a sentinel just like the basic setup documented by redis (http://redis.io/topics/sentinel). I've even created some simple sh scripts and template config/init files to automate setting up a redis or sentinel instance. Each node in the cluster is running RHEL 6 with 4 IPs assigned. One assigned to eht0, one assigned to eth0:1, one assigned to eht0:2, etc.. My first round of setting this up and testing it out went great with the each instance bound to the eth0 IP address. The sentinel works as it should when I stop the master and performs the failover, promoting one of the slaves and rewriting configs. However, when I setup the redis instance to be bound to any of the other IPs on a node, that's when things go awry. The redis instances themselves run fine, but when I setup the sentinels to monitor the master that is running on eht0:x, the log gets flooded with messages (several per second). When the sentinel first starts, it logs the usual info on the master, slaves, and other sentinels like so:

54370:X 16 Oct 15:43:13.331 # Sentinel runid is 0c1c80343abb3803787bd77caff6e5ad33ba7a67
54370:X 16 Oct 15:43:13.331 # +monitor master test 10.64.81.51 6380 quorum 2
54370:X 16 Oct 15:43:13.331 * +slave slave 10.65.80.120:6380 10.65.80.120 6380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:13.341 * +slave slave 10.20.32.71:6380 10.20.32.71 6380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:15.342 * +sentinel sentinel 10.20.32.70:26380 10.20.32.70 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:15.388 * +sentinel sentinel 10.65.80.120:26380 10.65.80.120 26380 @ test 10.64.81.51 6380

This then gets followed by the flood of messages as stated:

54370:X 16 Oct 15:43:15.391 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.65.80.119:26380 or 57811beba28c6e328b845fdc5c557fc846153ec4
54370:X 16 Oct 15:43:15.391 * +sentinel sentinel 10.65.80.119:26380 10.65.80.119 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:15.397 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.20.32.71:26380 or b2b7b2d4aab9471be773d94034bf6ddc94c09c73
54370:X 16 Oct 15:43:15.397 * +sentinel sentinel 10.20.32.71:26380 10.20.32.71 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:15.400 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.20.32.70:26380 or b2b7b2d4aab9471be773d94034bf6ddc94c09c73
54370:X 16 Oct 15:43:15.400 * +sentinel sentinel 10.20.32.70:26380 10.20.32.70 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:17.400 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.65.80.120:26380 or 57811beba28c6e328b845fdc5c557fc846153ec4
54370:X 16 Oct 15:43:17.401 * +sentinel sentinel 10.65.80.120:26380 10.65.80.120 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:17.403 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.65.80.119:26380 or 57811beba28c6e328b845fdc5c557fc846153ec4
54370:X 16 Oct 15:43:17.403 * +sentinel sentinel 10.65.80.119:26380 10.65.80.119 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:17.446 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.20.32.71:26380 or b2b7b2d4aab9471be773d94034bf6ddc94c09c73
54370:X 16 Oct 15:43:17.446 * +sentinel sentinel 10.20.32.71:26380 10.20.32.71 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:17.529 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.20.32.70:26380 or b2b7b2d4aab9471be773d94034bf6ddc94c09c73
54370:X 16 Oct 15:43:17.529 * +sentinel sentinel 10.20.32.70:26380 10.20.32.70 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:19.403 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.65.80.120:26380 or 57811beba28c6e328b845fdc5c557fc846153ec4
54370:X 16 Oct 15:43:19.403 * +sentinel sentinel 10.65.80.120:26380 10.65.80.120 26380 @ test 10.64.81.51 6380
54370:X 16 Oct 15:43:19.405 * -dup-sentinel master test 10.64.81.51 6380 #duplicate of 10.65.80.119:26380 or 57811beba28c6e328b845fdc5c557fc846153ec4

I've seen other posts to this group with similar messages being logged, but they haven't gotten a response yet. I'm hoping providing a little more information will help. I will attach a copy of my configs in case that would help. Has anyone come across this issue and figured out a way around it?

Thanks for any help you might have.

Regards,

Ben

[Mike Campbell]

I too am having this issue with multiple interfaces; did you find a solution, Ben?