[WARN] memberlist: Was able to connect co consul-1 but other probes failed, ...

[Abdulkadir Dalga]

Hello everyone,

I have deployed 3 node consul cluster on Openshift Container Platform. I run the nodes with following args.

agent -server -advertise=172.30.107.51 -ui -bootstrap-expect=3 -client=0.0.0.0 -data-dir=/var/lib/consul -config-dir=/var/lib/consul/config -retry-join=172.30.69.1 -retry-join=172.30.89.254 -node=consul-1

agent -server -advertise=172.30.69.1 -ui -bootstrap-expect=3 -client=0.0.0.0 -data-dir=/var/lib/consul -config-dir=/var/lib/consul/config -retry-join=172.30.107.51 -retry-join=172.30.89.254 -node=consul-2

agent -server -advertise=172.30.89.254 -ui -bootstrap-expect=3 -client=0.0.0.0 -data-dir=/var/lib/consul -config-dir=/var/lib/consul/config -retry-join=172.30.69.1 -retry-join=172.30.107.51 -node=consul-3

If I remove the -advertise argument there is no problem but the nodes talk each others over pods' IP and since the pods' IP are dynamic there is a possibility for outage scenario. To prevent this I tried to advertise services' IP. It prevents outage scenario and seems work well; however, when I restart 1 node the other nodes logs warning like [WARN] memberlist: Was able to connect to consul-1 but other probes failed,  network may be misconfigured . 

1- Is this a problem?

2- Can we solve this ?

Regards,

Abdulkadir Dalga

[James Phillips]

Hi,

Sorry for the late reply on this one - did you figure it out? I'm not
I follow what you mean by "I tried to advertise services' IP". Also,
in Consul 0.9.3 we did some work to help Consul servers recover when
their IPs change after a restart, which might also help:

> server: Consul servers can re-establish quorum after all of them change their IP addresses upon a restart. [GH-1580]

(from https://github.com/hashicorp/consul/blob/v0.9.3/CHANGELOG.md).

-- James

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/consul/issues
> IRC: #consul on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Consul" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to consul-tool...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/consul-tool/3e76f150-f430-4415-80db-913395452fd1%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.