etcd read wal error (walpb: crc mismatch)

1,426 views
Skip to first unread message

eldoran stars

unread,
Aug 28, 2022, 8:25:48 AM8/28/22
to etcd-dev
An unexpected power outage caused my kubernetes cluster's etcd to crash at startup. 
Etcd on master-1 node give error:
Aug 28 09:59:58 master-1 etcd[2670]: advertise client URLs = https://192.168.5.11:2379 
Aug 28 09:59:58 master-1 etcd[2670]: read wal error (walpb: crc mismatch) and cannot be repaired 
Aug 28 09:59:58 master-1 systemd[1]: etcd.service: Main process exited, code=exited, status=1/FAILURE 
Aug 28 09:59:58 master-1 systemd[1]: etcd.service: Failed with result 'exit-code'.
I try to follow this instruction:
https://etcd.io/docs/v3.5/tutorials/how-to-deal-with-membership/
But cannot connect:
vagrant@master-1:~$ sudo ETCDCTL_API=3 etcdctl member list \ 
> --endpoints=https://127.0.0.1:2379
> --cacert=/etc/etcd/ca.crt \ 
> --cert=/etc/etcd/etcd-server.crt \ 
> --key=/etc/etcd/etcd-server.key 
Error: dial tcp 127.0.0.1:2379: connect: connection refused
How can I fix walpb: crc mismatch on etcd master-1 node?

P.S.:

eldoran stars

unread,
Aug 28, 2022, 11:09:03 AM8/28/22
to etcd-dev
sudo cp /var/lib/etcd/member/wal/0000000000000000-0000000000000000.wal wal-bu
sudo rm /var/lib/etcd/member/wal/0000000000000000-0000000000000000.wal
sudo systemctl restart etcd.service
It works now!

воскресенье, 28 августа 2022 г. в 15:25:48 UTC+3, eldoran stars:
Reply all
Reply to author
Forward
0 new messages