Alertmanager silence disappeared after container restart

36 views
Skip to first unread message

Dovid Friedman

unread,
Feb 8, 2022, 3:28:24 AM2/8/22
to Prometheus Users
I have Alertmanager running in K8S and every time I restart the pod all my silences disappear  

Brian Candler

unread,
Feb 8, 2022, 3:59:34 AM2/8/22
to Prometheus Users
Alertmanager stores its state in disk files, by default in subdirectory "data" relative to the working directory where alertmanager was launched.

From alertmanager --help:

      --storage.path="data/"     Base path for data storage.
      --data.retention=120h      How long to keep data for.


If you don't preserve that disk file, then it will lose its state.  If this is in kubernetes then you'll need to use a PVC to attach some sort of persistent storage volume.

Dovid Friedman

unread,
Feb 8, 2022, 8:36:58 AM2/8/22
to Prometheus Users
Hi
I added a PVC and have /data in there 
thanks you 

Reply all
Reply to author
Forward
0 new messages