ETCD Wal files growing, slowly getting purged and DB size is very small

[vwalek]

Hello,
I have a situation when running etcd 3.2 (old cluster) where the WAL files are growing (up to ~7G) of total size, not being purged (very slowly) and still around 100-200 files stay on the disc.

What is more curious is that the DB size is only ~25KB.

My assumption is that the data are not being saved in the DB file. The fsync and disk latency are fine.

Any idea what could cause that? No errors shown in the cluster, no alarm present.

[jmhbnz]

Hi @vwalek - As you mentioned etcd 3.2 is quite old, etcd supported releases are currently 3.4 and 3.5 so please upgrade if you can: https://etcd.io/docs/v3.5/op-guide/versioning

One quick thing that you could check is your cluster ETCD_MAX_WALS setting in etcd.conf and ensure this is not set to 0 as doing so will mean that WAL files will never be purged.