Time Interval - Rules.yml -alert manager

119 views
Skip to first unread message

vamsi madhav

unread,
Apr 16, 2021, 2:22:54 PM4/16/21
to Prometheus Users
My requirement is to check the expression three times before it send alert in slack or Email 

Current Config: 
In the below config, for:5m will send an alert if the memory usage is over 80% . But I want to check the expression vmware_vm_mem_usage_average / 100 >= 80 in loop which means when the Memory is over 80% the expression will counter first time and recheck with same expression for second time and again recheck third time and when the memory is still over 80% after three checks then the alert will start .
 
groups:
  - name: target
    rules:
      - alert: VirtualMachineMemoryWarning
        expr: vmware_vm_mem_usage_average / 100 >= 80
        for: 5m  
        labels:
          severity: warning
          team: devops
        annotations:
          summary: Virtual Machine Memory Warning (instance {{ $labels.instance }})
          description: High memory usage on {{ $labels.instance }}

Stuart Clark

unread,
Apr 16, 2021, 5:44:26 PM4/16/21
to vamsi madhav, Prometheus Users
What you are asking for is exactly what the "for" option does. It keeps periodically evaluating the expression and will only fire the alert if it constantly returns a result.

You can set how often those checks happen, so for example if they were every minute "for: 5m" means 5 evaluations all have to return the same time series to allow an alert to fire.
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Reply all
Reply to author
Forward
0 new messages