Any way to restore my data?

[John Ruiz]

An unexpected power outage caused my kubernetes cluster's etcd to crash at startup, complaining about "wal: crc mismatch".

I copied the database from /var/lib/etcd/member/snap/db and then deleted the /var/lib/etcd directory and attempted to restore from the db:

ETCDCTL_API=3 ./etcdctl snapshot restore ../db --name compute --initial-cluster compute=https://10.11.12.2:2380 --initial-cluster-token etcd-cluster-1 --initial-advertise-peer-urls https://10.11.12.2:2380 --skip-hash-check=true --data-dir=/var/lib/etcd

But this fails with:

{"level":"panic","ts":1640679463.064161,"caller":"mvcc/key_index.go:82","msg":"'put' with an unexpected smaller revision","given-revision-main":37158302,"given-revision-sub":0,"modified-revision-main":37159152,"modified-revision-sub":0,"stacktrace":"go.etcd.io/etcd/mvcc.(*keyIndex).put\n\t/home/remote/sbatsche/tmp/etcd-new-release/release-3.4/etcd/release/etcd/mvcc/key_index.go:82\ngo.etcd.io/etcd/mvcc.restoreIntoIndex.func1\n\t/home/remote/sbatsche/tmp/etcd-new-release/release-3.4/etcd/release/etcd/mvcc/kvstore.go:507"}

I do not understand how etcd works, so I don't understand what "revisions" are, or why a smaller version is unexpected and causes a panic.

Is there anything I can do to get a new etcd instance running with my old data?

[tangc...@gmail.com]

Are you using etcd v3.4.8？upgrade to v3.4.9 or other latest version to restart.

[John Ruiz]

I should have specified my version, I am sorry.  

This is 3.4.13

--
You received this message because you are subscribed to a topic in the Google Groups "etcd-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/etcd-dev/9j7qxtMkrDw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to etcd-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/etcd-dev/38f8619e-630c-43e8-8db9-19c888322763n%40googlegroups.com.