Is it possible to function with a minority?

64 views
Skip to first unread message

Ex Puexto

unread,
Nov 4, 2021, 9:39:42 AM11/4/21
to raft-dev
Hello!

I'm new to Raft.
I'm trying to figure out whether it is suitable for my application, before I dive deeper.

I'm working on an embedded application with 4 "servers" which may or may not be on, and may or may not communicate with each other. Whichever servers are on and visible have to agree to a "state".

Is it possible, in case [less than] half of the servers are on, to choose a leader and agree to a state?

Maybe an initial cluster of 2 can be defined, and change the cluster configuration when more servers are available?

Any suggestions are welcome!

Thank you to all in advance!

Ex


Philip O'Toole

unread,
Nov 4, 2021, 9:46:50 AM11/4/21
to raft...@googlegroups.com
On Thu, Nov 4, 2021 at 9:39 AM Ex Puexto <expu...@gmail.com> wrote:
Hello!

I'm new to Raft.
I'm trying to figure out whether it is suitable for my application, before I dive deeper.

I'm working on an embedded application with 4 "servers" which may or may not be on, and may or may not communicate with each other. Whichever servers are on and visible have to agree to a "state".

Is it possible, in case [less than] half of the servers are on, to choose a leader and agree to a state?

Maybe an initial cluster of 2 can be defined, and change the cluster configuration when more servers are available?
 
Yes, this is possible. rqlite, which uses Hashicorp's Raft implementation, supports this. Servers can be added and removed at any time. However to make these changes, a quorum of servers -- as it is configured at the time the change is made -- must be up and running.



Any suggestions are welcome!

Thank you to all in advance!

Ex


--
You received this message because you are subscribed to the Google Groups "raft-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raft-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/raft-dev/6ac31795-28b8-4cbf-afd1-07ce36c7bf20n%40googlegroups.com.

Jesper Lindholm

unread,
Nov 4, 2021, 9:48:07 AM11/4/21
to raft...@googlegroups.com

Choosing a small cluster size and then growing is possible. You should avoid an even number of nodes for your clusters.

 

Taking your bolded question more broadly, this situation, to the involved servers, is indistinguishable from the situation where there is a networking or other error to partition the cluster, and both minority groups carry on the cluster separately, leading to two completely different versions of history.

 

Raft is made for live clusters where all nodes can communicate with each other reasonably quickly. Some of the fundamental properties depend on this. To handle a situation where normal operation means being off the network or not being in guaranteed contact with everyone else, I think you are better off looking for a sync/CDRT solution.

 

As an example, RavenDB is a database product using Raft for its clusters, but is also in use in your type of scenario, and it is my understanding that it has to use different technology to make this happen.

 

Regards,

/Jesper

--

Philip O'Toole

unread,
Nov 4, 2021, 9:50:48 AM11/4/21
to raft...@googlegroups.com
On Thu, Nov 4, 2021 at 9:48 AM Jesper Lindholm <Jesper....@treetop.se> wrote:

Choosing a small cluster size and then growing is possible. You should avoid an even number of nodes for your clusters.


Jesper Lindholm

unread,
Nov 4, 2021, 9:53:30 AM11/4/21
to raft...@googlegroups.com

Sorry, I misspelled ”CRDT”.

 

/Jesper

Ex Puexto

unread,
Nov 4, 2021, 12:23:03 PM11/4/21
to raft-dev
Thank you for your answers!
I see it's possible, but not straightforward.

I'll have a look, and let you know what we decide.

Thanks!

Jesper Lindholm

unread,
Nov 4, 2021, 12:34:58 PM11/4/21
to raft...@googlegroups.com

To rephrase what I said before to make the point better:

 

Raft (and other cluster consensus algorithms) is great if you have a cluster of nodes that always can communicate with each other and need to proceed in lockstep, but where for availability and robustness you want to gracefully handle nodes failing.

 

However, if one node can be off doing its own thing, maybe needs to work offline and sporadically gets to see the other nodes and synchronize what it has been doing, this requires an entirely different set of solutions. If half of the nodes can be unreachable at any given time, that’s not a workable cluster. It doesn’t mean you can’t solve your problem, it just means you’ll have to pick another way of solving it. CRDTs and offline sync algorithms are built for this and Raft isn’t.

sahar Al-Taee

unread,
Jan 7, 2022, 1:47:55 PM1/7/22
to raft-dev
hi 
Can we cooperate in the construction of the Raft project?

Reply all
Reply to author
Forward
0 new messages