Fire a specific alert once a day

[Shivam Chawla]

Hello Everyone, 

I am new to Prometheus and Alertmanager. I have a alert.rules where I have 95 alerts defined for which I get notified as per alertmanager configuration (alertmanager.yml)

My goal is to check for a specific alert once in 24 hours and then fire the alert if condition is true and then repeat it once a day. 

I understand the answer lies around scrape_interval in prometheus configuration (for checking once in 24 hours) and routes, repeat_interval in alertmanager configuration. 

Example: 

Alert rule:

ALERT ssl_cert_expire_critical

  IF probe_ssl_earliest_cert_expiry{job="blackbox"} - time() < 86400 * 30

  FOR 5m

  ANNOTATIONS {

    summary="SSL cert for the site {{ $labels.instance }} has less than 30 days until it expires",

    severity="critcal"

}

As per this alert rule, could you please help me with the job configuration which I can put in my prometheus.yml file and also the alertmanager configuration which I can put in my alertmanager.yml file

Thank you!

-Shivam Chawla

[lroch...@leapfrogonline.com]

I achieved this by creating a label called "frequency" that I set to daily on the alert. Then I created a separate route that matched any alerts with frequency = daily. Example:

in my alert...

labels:

        severity: warning
        audience: "some_group"
        frequency: daily

alertmanager.yml

  routes:
    # alerts that we only want to notify once per day
    - receiver: slack
      match:
        frequency: daily
      continue: false
      repeat_interval: 24h

   - receiver: email
      match:
        frequency: daily
      continue: false
      repeat_interval: 24h

 

[vidhigu...@gmail.com]

Hello

Is there a way to ensure that the alert is triggered at a specific time?

Thank You

Vidhi Gupta

[Julien Pivotto]

On 24 Apr 00:28, vidhigu...@gmail.com wrote:
> Hello
>
> Is there a way to ensure that the alert is triggered at a specific time?
>
> Thank You
> Vidhi Gupta

my_promql_rule and on () vector(hour() == 10)

>
> On Saturday, May 5, 2018 at 2:59:00 AM UTC+5:30,
> lroch...@leapfrogonline.com wrote:
> >
> > On Monday, April 23, 2018 at 12:17:19 PM UTC-5, Shivam Chawla wrote:
> >>
> >> Hello Everyone,
> >>
> >> I am new to Prometheus and Alertmanager. I have a alert.rules where I
> >> have 95 alerts defined for which I get notified as per alertmanager
> >> configuration (alertmanager.yml)
> >> My goal is to check for a specific alert once in 24 hours and then fire
> >> the alert if condition is true and then repeat it once a day.
> >>

> >> I understand the answer lies around *scrape_interval* in prometheus
> >> configuration (for checking once in 24 hours) and *routes*,
> >> *repeat_interval* in alertmanager configuration.
> >>
> >> *Example:*
> >>
> >> *Alert rule:*

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To post to this group, send email to promethe...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/322c02a9-2720-4dac-ba28-2518d557790b%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


--
(o- Julien Pivotto
//\ Open-Source Consultant
V_/_ Inuits - https://www.inuits.eu

[Amit Das]

Hi 

i got similar issue i would like to get alert only once a day at specific time. Already added alertmanager  routes as mentioned above. I tried with my rule but not query is wrong for scalar got instant vector.

(max_over_time(probe_sucess[20h]) == 0 ) and on () vector(hour() == 10)

Please suggest.

[Amit Das]

hi

My querry looks wrong as i am getting vector error. Please correct this where i am doing wrong.

(max_over_time(probe_sucess[20h]) == 0 ) and on () vector(hour() == 10)

Thanks