Question about snapshot in copycat

[Terry Tan]

Hi Jodan,

how often is the snapshot taking in copycat, i found every time  when appendRPC sends ,it will check the index ,if it is greater than another member ,it will send install snapshot. will it be too frequent? 

[jordan.h...@gmail.com]

So, there are a couple of things to note here. You're right, in Copycat a leader checks if its snapshot index is greater than the follower's last index and sends the snapshot of so. That's because once a leader takes a snapshot, it marks SNAPSHOTTABLE entries prior to that snapshot to be deleted from the log and will no longer replicate them, so the snapshot is replicated in its place.

Sure, at first glance it seems entirely possible a leader in a three node cluster could replicate to one other node, take a snapshot, and then have to send that snapshot to the third node, which would typically be considered inefficient. But it's important to note that snapshots are not Copycat's primary log compaction mechanism. Rather, snapshots are supplementary to its incremental compaction algorithm. They more or less serve to satisfy use cases for which incremental compaction is impractical or impossible, like counters. So, snapshot state will typically be very small since the majority of entries are compacted using the incremental compaction API, so sending a snapshot in Copycat is usually cheap.

Also, snapshots in Copycat are only taken at the end of segments and aren't necessarily considered complete when the snapshot is taken. There's usually some latency between when the snapshot is written by the state machine and when it's actually committed to disk and can be replicated to followers. This latency is because Copycat will never snapshot entries on disk that trigger session events, otherwise it would have to snapshot the events themselves. Instead, it takes a snapshot at a specific point in the log and writes the snapshot to disk, but it waits for any events that were triggered prior to the snapshot to be acknowledged by clients so it's safe to delete those entries.

While it's unlikely to result in snapshots being sent too often, I think that condition could still be improved a bit. It could realistically check if any entries have been compacted between the end of the follower's log and the leader's snapshot and not send the snapshot unless entries have actually been compacted.

Thanks for pointing that out!

On Oct 21, 2016, at 4:19 AM, Terry Tan <tx...@sina.com> wrote:

Hi Jodan,

how often is the snapshot taking in copycat, i found every time  when appendRPC sends ,it will check the index ,if it is greater than another member ,it will send install snapshot. will it be too frequent? 

--
You received this message because you are subscribed to the Google Groups "raft-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raft-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.