Alertmanager is not deduplicated.

207 views
Skip to first unread message

masanori suzuki

unread,
Aug 29, 2019, 9:52:28 PM8/29/19
to Prometheus Users
hi.

Clustered with two alertmanagers.

Prometeus also consists of two units.

prometheus has an external label and the settings are the same except that the value of replica is different.

Isn't alertmanager deduplication based on the label specified in group_by?
Since the replica label value is different, two notifications have been received, and I want to know if deduplication is possible only with the label specified in group_by.


The following definition is made in alertmanger.yml.
---
  group_by: [region, alertname, instance, job]
  group_wait: 1s
  group_interval: 1m
  receiver: test
  routes:
    -match:
        job: icmp
      receiver: test
---

・PrometeusA
prometheus.yml has the following definition:
---
  external_labels:
    region: us-east-1
    replica: a
    
  -job_name: icmp
    metrics_path: / probe
    params:
      module: [icmp]
    file_sd_configs:
      -files:
          -/usr/local/prometheus/config/targets/icmp_*.yml
    relabel_configs:
      -source_labels: [__address__]
        target_label: __param_target
      -source_labels: [__param_target]
        target_label: instance
      -target_label: __address__
        replacement: test.example.com
---

・PrometeusB
prometheus.yml has the following definition:
---
  external_labels:
    region: us-east-1
    replica: b
    
  -job_name: icmp
    metrics_path: / probe
    params:
      module: [icmp]
    file_sd_configs:
      -files:
          -/usr/local/prometheus/config/targets/icmp_*.yml
    relabel_configs:
      -source_labels: [__address__]
        target_label: __param_target
      -source_labels: [__param_target]
        target_label: instance
      -target_label: __address__
        replacement: test.example.com
---

Brian Brazil

unread,
Aug 30, 2019, 6:02:27 AM8/30/19
to masanori suzuki, Prometheus Users
On Fri, 30 Aug 2019 at 02:52, masanori suzuki <masuzu...@gmail.com> wrote:
hi.

Clustered with two alertmanagers.

Prometeus also consists of two units.

prometheus has an external label and the settings are the same except that the value of replica is different.

Isn't alertmanager deduplication based on the label specified in group_by?

No, deduplication is based on all labels. Grouping is based on the group_by.


Brian
 
--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/67c8ff3b-6bd5-44e4-9576-4c53c7aa975b%40googlegroups.com.


--

masanori suzuki

unread,
Aug 31, 2019, 5:01:51 AM8/31/19
to Brian Brazil, Prometheus Users
Thank you for your reply.
I understand that deduplication requires that labels and values match exactly.

2019年8月30日(金) 19:02 Brian Brazil <brian....@robustperception.io>:
Reply all
Reply to author
Forward
0 new messages