need help in relabeling

37 views
Skip to first unread message

ishu...@gmail.com

unread,
Mar 17, 2021, 12:07:46 PM3/17/21
to Prometheus Users
Hello,

From my prometheus, i am monitoring a storage system.I have an issue with the alerts coming out of this storage monitoring.

I have my alert manager grouping alerts by instance. the storage system is a single instance but it has directories for each application and alerts are in place on the total space used by these apps wrt to allocated space. The alerts that are triggered when the usage reaches threshold is being group and i am only receiving one instance but I want to receive multiple alerts for each directory (referred as 'name' in the label).

I don't want to add this(name) to the group_by in alert manager as this would distort another infra monitoring alerts in place. 

My idea is to set intance as combination of instance and directoryname but doesn't like this approach much. While recording might help, but i am unable to relabel labels in recording rules. 

 - job_name: 'pure_flashblade'
    scrape_interval: 60s
    scrape_timeout: 30s
    metrics_path: /metrics/flashblade
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_endpoint
      - source_labels: [__pure_apitoken]
        target_label: __param_apitoken
      - source_labels: [__address__]
        target_label: instance
        replacement: ${1}_${name}
      - target_label: __address__
        replacement: pure-exporter:9491
    static_configs:
      - targets: ["pure-storage.com"]
        labels:
          __pure_apitoken: "${PURE_API_TOKEN}"
          env: "prod"

Any help is much appreciated.

Thanks
Eswar

Julien Pivotto

unread,
Mar 17, 2021, 1:13:57 PM3/17/21
to ishu...@gmail.com, Prometheus Users
You can use:

- source_labels: [__address__, name]
  separator: _
  target_label: instance

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/1b6199ba-91a7-4a22-ad7b-f3755b3853b1n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages