Alertmanager status component=silences msg="Running maintenance failed" permission denied

69 views
Skip to first unread message

Annonyme1

unread,
Sep 11, 2020, 11:41:16 AM9/11/20
to Prometheus Users
I was working with alertmanger and it was sending alert after a while i got that error : 
alertmanager[15768]: level=info ts=2020-09-11T15:14:54.414Z caller=silence.go:379 component=silences msg="Running maintenance failed" err="open data/silences.16bdb0fb9c4ccc4e: permission denied"


Some help please !!

Brian Candler

unread,
Sep 12, 2020, 4:13:59 AM9/12/20
to Prometheus Users
I'm stating the obvious here, but it looks like a permissions problem on that directory or file.

At minimum you should:

1. identify what uid alertmanager is running as, and what working directory it's running in
2. check permissions on the "data" subdirectory, and all files in that directory

Annonyme1

unread,
Sep 17, 2020, 4:10:13 AM9/17/20
to Prometheus Users
I did it with chmod 777 but after restarting 777 i the autorisations gaven earlier and i got the same message  :"" component=silences msg="Running maintenance failed" err="open data/silen...sion denied""""

Brian Candler

unread,
Sep 17, 2020, 6:27:29 AM9/17/20
to Prometheus Users
Permissions errors can also come from SELinux or Apparmor.  Check for errors in the system logs.  If it's a CentOS/RedHat system, try setting SELinux to permissive mode.

Annonyme1

unread,
Sep 23, 2020, 11:41:02 AM9/23/20
to Prometheus Users
now i got that error """component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="dial tcp grafana.s.com: connect: no route to host; dial tcp grafana.s.com:25: connect: no route to host""""

Brian Candler

unread,
Sep 23, 2020, 1:08:32 PM9/23/20
to Prometheus Users
That's a very clear error message.  Alertmanager is trying to make a connection to grafana.s.com port 25 (SMTP for sending mail), and it can't.  Possibly the connection is being blocked by firewall rules.

You can test it by hand:

telnet grafana.s.com 25

and then fix your network until it works.
Reply all
Reply to author
Forward
0 new messages