Persistent commit index if state machine is persistent?

[Алексей Корчевский]

Hello!

I'm working on raft implementation with persistent state machines, but I'm confused reading chapter 3.8

Other state variables are safe to lose on a restart, as they can all be recreated. The most interesting example is the commit index, which can safely be reinitialized to zero on a restart. Even if every server restarts at the same time, the commit index will only temporarily lag behind its true value. Once a leader is elected and is able to commit a new entry, its commit index will advance, and it will quickly propagate this commit index to its followers.

The state machine can either be volatile or persistent. A volatile state machine must be recovered after restarts by reapplying log entries (after applying the latest snapshot; see Chapter 5). A persis- tent state machine, however, has already applied most entries after a restart; to avoid reapplying them, its last applied index must also be persistent.

But wait - by oblivious rules, last applied index can't be less than committed index, right?

Why committed index doesn't persistent in this case, or maybe I should initialize committed index to last applied?

[Archie Cobbs]

The "commit index" is not the index of the last committed log entry, it's the index of the last known-to-be-committed log entry. So initializing to zero (which means you know nothing) on restart is always an appropriate choice.

If you are persisting your state machine, and therefore also the last applied index, you are persisting some additional, optional information that will allow you to know something extra on restart, because no log entry should ever get applied to the state machine unless it is known to be committed.

So - I'd say yes, such a node should be able to safely initialize its commit index to the last applied index instead of zero. However this is a small optimization that will probably not make much difference in practice, and only applies in certain cases, and was therefore probably not worth mentioning in the description quoted.

-Archie

On Thursday, April 13, 2017 at 4:39:49 AM UTC-5, Алексей Корчевский wrote:

I'm working on raft implementation with persistent state machines, but I'm confused reading chapter 3.8

Other state variables are safe to lose on a restart, as they can all be recreated. The most interesting example is the commit index, which can safely be reinitialized to zero on a restart. Even if every server restarts at the same time, the commit index will only temporarily lag behind its true value. Once a leader is elected and is able to commit a new entry, its commit index will advance, and it will quickly propagate this commit index to its followers.

The state machine can either be volatile or persistent. A volatile state machine must be recovered after restarts by reapplying log entries (after applying the latest snapshot; see Chapter 5). A persistent state machine, however, has already applied most entries after a restart; to avoid reapplying them, its last applied index must also be persistent.