Send message via Alertmanager to two slack channels

[g...@recongate.com]

Hello,

This is my alertmanager.yml file:

------------------------------------------------------------------------------------------

global:

  slack_api_url: '< slack_url >'

route:

  receiver: 'Monitorbot'

  # All alerts in a notification have the same value for these labels.

  group_by: [alertname, datacenter, app]

route:

  receiver: 'Production'

  # All alerts in a notification have the same value for these labels.

  group_by: [Env, alertname, datacenter, app]

receivers:

- name: 'Production'

  slack_configs:

  - channel: '#alerts'

    title: "{{ range .Alerts }}{{ .Annotations.summary }}\n{{ end }}"

    text: "{{ range .Alerts }}{{ .Annotations.description }}\n{{ end }}"

- name: 'Monitorbot'

  slack_configs:

  - channel: '#devops'

    title: "{{ range .Alerts }}{{ .Annotations.summary }}\n{{ end }}"

    text: "{{ range .Alerts }}{{ .Annotations.description }}\n{{ end }}"

------------------------------------------------------------------------------------------

My objective is to send alerts with the label {Env : 'production'} to the #alerts channel and all the alerts (including the ones with this label} to the #devops channel.

How should i change my configuration to achieve this?

[jdbj...@gmail.com]

Don't use the global slack_api_url, instead use te api_url under slack_configs. Configuring two integrations on slack, one per channel, you can then use the specific integration on the receiver you want.

https://prometheus.io/docs/alerting/configuration/#<slack_config>

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/7ad5a602-4ff1-4d9b-88dc-4f301830d0c0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gil Goldberger]

Thanks,

Do you have any more documentation for this or perhaps templates i could use?

[Gil Goldberger]

Also, where do i specify that alerts with the label {Env : 'production'} should be forwarded to a certain slack channel?

[jdbj...@gmail.com]

Alertmanager config documentation -> https://prometheus.io/docs/alerting/configuration

example:

route:

  receiver: webhook

  routes:

    - match:

        env: production

      receiver: slack-production

    - match:

        env: staging

      receiver: slack-staging

receivers:

- name: slack-production

  slack_configs:

  - api_url: https://hooks.slack.com/services/<XXX>

    channel: production

- name: slack-staging

  slack_configs:

  - api_url: https://hooks.slack.com/services/<XXX>

    channel: staging

- name: webhook

  webhook_configs:

  - url: webhook_url

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CALOMY4VOCZe7dCHcnmYMcd3roz05FT6U%3DP4aQ4a53%2BTSPTO3BQ%40mail.gmail.com.

[g...@recongate.com]

Hi,

This is my alertmanager.yml file:

global:

route:

  receiver: 'default'

  routes:

  - match:

      env: 'production'

      receiver: "Alerts-production"

  - match:

      env: 'development'

      receiver: "Alerts-development"

receivers:

  - name: 'Alerts-production'

    slack_configs:

    - api_url: ''< slack_url >'

      channel: '#alerts'

  - name: 'Alerts-development'

    slack_configs:

    - api_url: ''< slack_url >'

      channel: '#alerts-development'

  - name: 'default'

    slack_configs:

    - api_url: ''< slack_url >'

      channel: '#devops'

No matter what the value of the label env is, the alert always goes to the default receiver.

I add the env label in the prometheus.yml file, it looks like this:

  - job_name: 'Dev'

    scrape_interval: 15s

    scrape_timeout: 10s

    static_configs:

      - targets: ['<host>:<port>']

        labels: {'env': 'production'}

Why isn't the alert redirected to the relevant receiver through the env label?

Thanks

[Simon Pasquier]

I suspect that you have an indentation issue with in your YAML configuration file. The "receiver" key should be at the same level as "match":

[...]
route:
  receiver: 'default'
  routes:
  - receiver: "Alerts-production"
    match:
      env: 'production'
  - receiver: "Alerts-development"
    match:
      env: 'development'
[...]

--

You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/9c53a5cc-641c-473c-8a83-354920fe79ea%40googlegroups.com.

[g...@recongate.com]

That was the problem, Thank you!

[Ali Kahoot]

Hi, I also want to send alerts to multiple slack channels bases on the labels defined on pods and this is my config but it is not sending notifications. only the default route receiver is working, can anyone tell what am I missing:

global:


  resolve_timeout: 5m


templates:


- '/etc/alertmanager/template/*.tmpl'


route:


  group_by: ['alertname', 'cluster', 'service']


  group_wait: 30s


  group_interval: 5m


  repeat_interval: 1h


  receiver: 'default-receiver'

  routes:


  - match:


      env: 'production'

    receiver: 'alerts-receiver'


  - match:


      alertname: DeadMansSwitch


    receiver: 'null'


  - match:


      env: 'development'


    receiver: 'alerts-receiver'

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

[Simon Pasquier]

It is definitely possible to achieve this.

Can you share the AlertManager's log files?

Are you 100% sure that the alerts have the "env" label set with the expected value?

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/4a572c88-c4fe-45c6-8294-f1aa3c7f84e7%40googlegroups.com.