Consul IP advertising problems when DHCP-assigned IP changes

[John Sullivan]

Hello Consul maintainers,

I'm chasing a Consul node-flapping issue that is triggered by a change of IP address.

In our environment, development nodes running a Consul agent are configured to go into sleep or standby when not in use.  Unfortunately the DHCP server in this environment assigns a new IP address to each of these nodes when the node returns to a running state, which means the IP address that Consul advertises is no longer associated with the node on which the agent is running.  This data center contains a mix of Consul agent versions, anywhere between 0.5.0 and 0.5.2.

Has anybody else experienced this (or a similar) issue?  Does Consul offer any options to work around this type of scenario?

Thanks in advance for your help and time,

John

[Maxime Brugidou]

I would suggest to configure your nodes to properly leave consul (using the co sul leave command) before going to sleep. The agent can be configured to leave by default when stopped for example.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/29605a79-8f03-4941-b57d-32220cd8135e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Fischer]

You can usually configure dhclient(8) to restart Consul upon address reassignment.  The specific details depend on the distribution you're using.

For example, on Ubuntu, you can add a hook script in /etc/dhcp/dhclient-exit-hooks.d to do so.

--

[John Sullivan]

Thanks Michael.  I'll investigate this technique for our Linux-based systems and look for a similar mechanism for Windows.

John

[Craig Sawyer]

As a side note, I had an issue today where a consul agent was running on a host, and the IP address changed on the host.  Consul didn't notice, and kept saying the service was at the old IP address.  Consul kept talking, kept saying the service was up, but with the old IP address.  

Is there some way to have Consul behave a bit differently here?  I mean even just dying would be totally fine with me, since my process manager would restart it, and that would then force consul to see the new IP address.

My workaround for now is to have dhclient (as an exit hook) restart consul.

-Craig

[James Phillips]

Hi Craig I opened https://github.com/hashicorp/consul/issues/1580 to capture that last item.

--

This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/623398ba-1dee-4851-85a2-221ff539c355%40googlegroups.com.