My requirement is to check the expression three times before it send alert in slack or Email
Current Config:
In the below config, for:5m will send an alert if the memory usage is over 80% . But I want to check the expression vmware_vm_mem_usage_average / 100 >= 80 in loop which means when the Memory is over 80% the expression will counter first time and recheck with same expression for second time and again recheck third time and when the memory is still over 80% after three checks then the alert will start .
groups:
- name: target
rules:
- alert: VirtualMachineMemoryWarning
expr: vmware_vm_mem_usage_average / 100 >= 80
for: 5m
labels:
severity: warning
team: devops
annotations:
summary: Virtual Machine Memory Warning (instance {{ $labels.instance }})
description: High memory usage on {{ $labels.instance }}