Only Leader election - for scheduled jobs

[Ngm]

Hello,

I need to execute certain jobs in regular intervals (say every min). If a single node does this, we have a single point of failure. To avoid this, I was thinking of following scheme:

1. Nodes form a raft cluster, with leader election

2. Only the leader executes scheduled jobs

     2.1. Every node checks if it is the leader before executing jobs.

In order to achieve this only leader election is necessary. So is it possible that I only implement the leader election part of RAFT and achieve this? Are there are any problems with this approach?

[Oren Eini (Ayende Rahien)]

How long is a job, what happens if after the check, there is new leader?

Better to do this via a raft command. 

You have a set of nodes. Each minute they send a command "gimmie the job", the state machine select one of them for this minute.

That is consistent and stable.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "raft-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raft-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Philip O'Toole]

Agreed, you need to do this via a "command". The concept of a node "checking if it's the leader" is racy, unless you put it through the Raft subsystem. You could end up with two nodes executing a job that you only wanted executed once.

Check out this document, and an issue, related to racy leader checks in a raft-based system I built:

https://github.com/rqlite/rqlite/blob/master/DOC/CONSISTENCY.md

https://github.com/rqlite/rqlite/blob/master/DOC/CONSISTENCY.md

 

-----------------------------------------
http://www.philipotoole.com

To unsubscribe from this group and stop receiving emails from it, send an email to raft-dev+u...@googlegroups.com.

[Archie Cobbs]

To put it another way, if what you want from Raft is "consensus", then Raft can indeed give you that. But it doesn't do so via the leader election process; it does it via the log entry commit process.

-Archie

On Tuesday, January 16, 2018 at 11:49:40 AM UTC-6, Philip O'Toole wrote:

Agreed, you need to do this via a "command". The concept of a node "checking if it's the leader" is racy, unless you put it through the Raft subsystem. You could end up with two nodes executing a job that you only wanted executed once.

Check out this document, and an issue, related to racy leader checks in a raft-based system I built:

https://github.com/rqlite/rqlite/blob/master/DOC/CONSISTENCY.md

https://github.com/rqlite/rqlite/blob/master/DOC/CONSISTENCY.md

 

-----------------------------------------
http://www.philipotoole.com

On Monday, January 15, 2018 10:30 PM, Oren Eini (Ayende Rahien) wrote:

How long is a job, what happens if after the check, there is new leader?

Better to do this via a raft command. 

You have a set of nodes. Each minute they send a command "gimmie the job", the state machine select one of them for this minute.

That is consistent and stable.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Tue, Jan 16, 2018 at 2:39 AM, Ngm wrote:

Hello,

I need to execute certain jobs in regular intervals (say every min). If a single node does this, we have a single point of failure. To avoid this, I was thinking of following scheme:

1. Nodes form a raft cluster, with leader election

2. Only the leader executes scheduled jobs

     2.1. Every node checks if it is the leader before executing jobs.

In order to achieve this only leader election is necessary. So is it possible that I only implement the leader election part of RAFT and achieve this? Are there are any problems with this approach?

--
You received this message because you are subscribed to the Google Groups "raft-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raft-dev+unsubscribe@googlegroups.com.

[yihao yang]

I think you can use raft to do the leader election. You don't commit any log, so every node in the quorum can become the leader. But remember raft only make sure each term has exactly one leader. That means there is the possibility that multiple leaders exist at the same time, which will not satisfy your requirement.

What you need is to make sure in every single time, there only exists at most one leader. Here is my solution:

Form a lease between leader and slaves, let's say the lease time is T. A leader will turn to a candidate once no more than majorities' lease expired. The new leader has to wait for more than T before executing the periodical tasks.

I don't see this has any side effect on the raft protocol.

Thanks,

Yihao

在 2018年1月15日星期一 UTC-8下午4:39:00，Ngm写道：