etcd3 'incremental' snapshots

[Scott Clasen]

Hi All-

  There are several docs that make mention of 'incremental' snapshots in etcd3, and the fact that they provide desirable operational properties.

I had assumed that 'incremental' snapshots would be a snapshot with data 'since' some revision, but that appears not to be the case.

Looking at the .proto for the RPC api, the SnapshotRequest has no fields. https://github.com/coreos/etcd/blob/master/etcdserver/etcdserverpb/rpc.proto#L567

Can someone share some insight into what is meant when 'incremental' snapshots are discussed in the context of etcd3? Are they a thing?

Cheers and thanks!

SC

[anthony...@coreos.com]

Which docs? I see two mentions in Documentation/ but they're not really operational claims...

The incremental snapshots are part of the internal implementation for etcd. For v3, it's a batched transaction from applying the raft log; there's no way to access it (or really make sense of it) at the client level. The Snapshot RPC is for backing up the complete v3 boltdb database; it's used by the etcdctl snapshot commands.

For incrementally syncing / mirroring keys, the v3 client supports it in the mirror package (https://github.com/coreos/etcd/blob/master/clientv3/mirror) and it's available through etcdctl with the make-mirror command.

[Scott Clasen]

Aha, yeah that syncer looks more like what I was imaging. Thanks!

I guess the closest thing to 'docs' is http://thenewstack.io/coreos-updates-etcd-large-scale-container-coordination/ (sponsored by coreos). 

 

"it now provides incremental snapshots, which should further boost its reliability in high availability deployments"  

Some stuff in here too http://www.slideshare.net/CoreOS_Slides/etcd-mission-critical-keyvalue-store 

from there a googling of 'etcd3 incremental snapshots' leads to various hacker news and stackoverflows where random folks are giving explanations of why incremental snapshots are good, so I thought Id poke around the code and ask the dev list to make sure.

Cheers!

SC

[王豪]

Hi Scott

        I'm still puzzled about  the 'incremental snapshots' . Could you share your findings?

Thanks!

在 2016年12月1日星期四 UTC+8上午11:12:20，Scott Clasen写道：