Global Labels in Alerts

[goel....@gmail.com]

Hello Community, 

I have rules file created for alerts, and i want to defined global labels for my rules. Is it possible?? I have not got much info about it in Prometheus docs.

Below is my rules file:

groups:                                                                                                                                                                     

- name: ContainerAlerts

  rules:

  - alert: ContainerKilled

    expr: (sum by(name, instance) (time() - container_last_seen{id!="/"})) > 30

    for: 10s

    labels:

      datacenter: DC:Test                                                                                                                            severity: warning

    annotations:

      summary: "Container killed (instance {{ $labels.instance }})"

      description: "Container '{{ $labels.name }}' has disappeared from '{{ humanizeDuration $value }}' seconds on Node_ID: '{{ $labels.instance }}'"

[Brian Candler]

To add labels to *every* alert sent from this prometheus instance, see alert_relabel_configs:

https://prometheus.io/docs/prometheus/latest/configuration/configuration/#alert_relabel_configs

[goel....@gmail.com]

In the below code the highlighted part is hard coded. I am am not getting data-center as a label from Prometheus. I want to make it global as in there is no requirement to hard code this data-center label on every alert. With the above link it is not clear about the requirement. Can you please help here?

groups:                                                                                                                                                                     

- name: Container-alerts

  rules:

  - alert: Container-killed

    expr: (sum by(name, instance) (time() - container_last_seen{id!="/"})) > 30

    for: 10s

    labels:

      datacenter: DC:Test                                                                                                                           

      severity: warning

    annotations:

      summary: "Container killed (instance {{ $labels.instance }})"

      description: "Container '{{ $labels.name }}' has disappeared from '{{ humanizeDuration $value }}' seconds on Node_ID: '{{ $labels.instance }}'"

[Matthias Rampke]

As Brian said, use alert relabel configs, which are global, or set the datacenter as an external label (`external_labels` in the configuration file).

/MR

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/ab20e147-7073-4057-868f-f47dfca69d06n%40googlegroups.com.

[goel....@gmail.com]

Thanks Brian and Mattias.

[Ben Kochie]

This is the use case for external labels, they are attached to all alerts.

On Wed, Sep 9, 2020 at 12:44 PM Brian Candler <b.ca...@pobox.com> wrote:

To add labels to *every* alert sent from this prometheus instance, see alert_relabel_configs:

https://prometheus.io/docs/prometheus/latest/configuration/configuration/#alert_relabel_configs

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/9b3eade4-98ee-484e-b9a1-8a29f4e36e17o%40googlegroups.com.