Single-node k3s etcd failed to reconcile about k3s HOT 3 CLOSED

It sounds like your etcd datastore was corrupted. Did the k3s process crash, or did the OS experience a kernel panic and reboot? I've only ever seen anything like this when the node is unexpectedly restarted or powered off, and files are corrupted on disk. I will also note that raspberry pis are highly likely to experience filesystem corruption when using SD cards; we do not recommend using etcd with sd cards under any circumstances.

Removing the files from disk and restoring from a snapshot is the proper way to address datastore corruption. Since the errors are from etcd, and were triggered by a crash, I'm not sure we have anything to fix here in K3s.

from k3s.

It sounds like your etcd datastore was corrupted. Did the k3s process crash, or did the OS experience a kernel panic and reboot? I've only ever seen anything like this when the node is unexpectedly restarted or powered off, and files are corrupted on disk. I will also note that raspberry pis are highly likely to experience filesystem corruption when using SD cards; we do not recommend using etcd with sd cards under any circumstances.

Removing the files from disk and restoring from a snapshot is the proper way to address datastore corruption. Since the errors are from etcd, and were triggered by a crash, I'm not sure we have anything to fix here in K3s.

@brandond external usb/sata SSD, no corruption, system didn't reboot or panic, hence the issue.

from k3s.

Can you provide the complete logs from journald covering the time period before and after the crash? We don't have any way to handle etcd database file corruption in k3s, but if there is a preventable crash occurring we can take a look at that.

from k3s.