Change raft_protocol from 2 to 3 (Consul 1.0.0) without downtime?

618 views
Skip to first unread message

Edgaras Apšega

unread,
Nov 6, 2017, 5:36:02 AM11/6/17
to Consul
Hello,

We're upgrading our Consul clusters from 0.9.2 to 1.0.0, and the default raft_protocol version 3 is causing downtime. But upgrading Consul server nodes one-by-one with "raft_protocol 2" is working perfectly fine. How to upgrade cluster to use v3 without downtime? Changing configuration to "raft_protocol 3" and pushing changes to one-by-one server nodes and restarting them doesn't help. All nodes are still using protocol 2.



Preetha Appan

unread,
Nov 15, 2017, 9:45:56 AM11/15/17
to Consul
Hi 

I suspect that you are running into https://github.com/hashicorp/consul/issues/3611 which is fixed and will be part of Consul 1.0.1. As a workaround, I would suggest upgrading to 1.0 leaving it at raft_protocol 2, and then doing a rolling upgrade to 1.0.1 with raft protocol 3 at the same time.

Edgaras Apšega

unread,
Nov 21, 2017, 2:16:44 AM11/21/17
to Consul
Hey,

I just did ant upgrade on one of my clusters with 2 datacenters with raft_protocol 3 option. What's interesting that there was a part of a second of a downtime while querying DNS interface when restarting the last node.

Anyway, `consul members -wan` command shows that the protocol version is till 2, but I started seeing `raft: Added peer 32ae761a-a85b-2608-4fc1-6a2976714b3a, starting replication` like messages in logs, so it means that the cluster is using protocol 3 now?

Thomas

unread,
Nov 21, 2017, 2:22:09 AM11/21/17
to consu...@googlegroups.com

Hi Edgaras,

You may want to use : consul operator raft list-peers

consul members displays the gossip protocol version in use - I think.

Thomas

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/038ea884-8da0-48e0-98be-e3c87db177b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Edgaras Apšega

unread,
Nov 27, 2017, 2:07:55 AM11/27/17
to Consul
Thanks Thomas! I see that the RaftProtocol 3 is now in use.
Reply all
Reply to author
Forward
0 new messages