multiple notifications integration alert

[Laetitia Gilet]

Hello,
I use prometheus/alertmanager to monitor a ceph octopus cluster.  By default alertmanager send alerts to dahboards of three monitor node via a webhook integration.
I want all my alert define in prometheus to be sent by mail and to the ceph dashbord so I've simply added a email_configs part to the ceph-dashbord receiver as follow :

global:
  resolve_timeout: 5m

route:
  receiver: 'default'
  routes:
    - group_by: ['alertname']
      group_wait: 10s
      group_interval: 10s
      repeat_interval: 1h
      receiver: 'ceph-dashboard'

receivers:
- name: 'default'
  webhook_configs:
- name: 'ceph-dashboard'
  webhook_configs:
  - url: 'https://ceph-m01.storage.mydomain.com:8443/api/prometheus_receiver'
  - url: 'https://ceph-m03:8443/api/prometheus_receiver'
  - url: 'https://ceph-m02:8443/api/prometheus_receiver'
  email_configs:
  - send_resolved: false
  - to: admin.s...@mydomain.com
  - from: ce...@mydomain.com
  - auth_username: admin.s...@mydomain.com
  - hello: localhost
  - smarthost: smtp.mydomain.com:25
  - require_tls: false


This do not work and I have a "network error" message when i go to alertmanager url.
Could you help me to keep one receiver and two notifications integrations please ?

Laetitia

[Ian Billett]

Hello Laetitia,

It looks like your configuration is roughly correct per the alertmanager receiver documentation. I always recommend people to check out prometheus' amtool, which allows you to verify your alertmanager configuration locally, and test which receivers an alert will be routed through.

If you are saying that you cannot even access the alertmanager URL, that is a different problem from configuring your alerting rules.

Please remember when reaching out to this mailing list, the more information you can provide: logs, screenshots, extra information, the better that we can help you out.

Best,

Ian

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/44c2eee7-ff78-4db9-a515-2b391d265848n%40googlegroups.com.

[Laetitia Gilet]

Hello, Ian thanks for your reply,

alertmanager is running inside a container based on SLES 15 SP2, image prometheus-alertmanager 0.16.2 "installed via cephadm"

when i run podman exec <id container alertmanager> amtool --version, I read version 0.16.2 and amtool -h is ok but amtool check-config is not recognized

I have no  log with podman logs <ID container>

When I 've tried the config sent before I see on the prometheus url status page :

I'm not a container expert so I don't know where to find other logs

Laetitia

[Ian Billett]

Laetitia,

I'm afraid that I've not come across this "Network Error" issue before, let's hope someone else in this group might have some more insight - is there anything obvious in your logs?

I would also recommend copying that configuration to you local machine, and then running amtool against it.

Best,

Ian

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f97d6864-064c-4ea2-b2f5-861167566c04n%40googlegroups.com.