Remote peer does not have local node

[Pablo Fischer]

Howdy!

We were trying to upgrade one of our hosts from old hardware to new
hardware, the process we followed was:

1. Stop consul on host C.
2. power off the host
3. Change DNS to new IP.
4. Bring back host C.

Obviously, we lost leader election (this is a small cluster, of 3
peers) but we were "OK" with it. However once C host comes back, we
see the following log entries on host C:

2015/11/24 01:54:37 [WARN] raft: Remote peer A:8300 does not have
local node C:8300 as a peer
2015/11/24 01:54:37 [WARN] raft: Remote peer B:8300 does not have
local node C:8300 as a peer

The peers.json is identical (only 3 hosts) on the 3 hosts, so not sure
where that error could be coming up.

$ consul version
Consul v0.5.2

Thanks!
--
Pablo

[James Phillips]

Hi Pablo,

If the IP changed for host C I don't think you'll be able to just swap that in the peers.json file, as the replayed Raft log will still reference the old IP address. You'll need to stop all the servers, remove C from peers.json on A and B, start C with an empty peers.json, and then do a `consul join` command on C to join it to A or B.

There are details here in the "Failure of a Server in a Multi-Server Cluster" section:

https://www.consul.io/docs/guides/outage.html

-- James

--
Pablo

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/CAJRPuoxJxWEe%2BDDZEkv_2e0iJ8BK7jQo8uBTtZBb1uhzgw-FMA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Pablo Fischer]

Yep. That is what happened and noticed that the old C host was not fully retired so now two hosts were taking to consul and each one was claiming to be C. Once I stopped the old C it just all worked (tho I needed to bootstrap one of the hosts)

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/CAGoWc057oQ5gnweKyhTAn2fHJMOimu9e_8t24dd6iX5sEUvRUg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
Pablo