Prometheus dynamic alert rule.

91 views
Skip to first unread message

akshay sharma

unread,
Apr 17, 2021, 5:41:44 AM4/17/21
to Prometheus Users
In my setup, Prometheus is monitoring multiple nodes simultaneously, say, x,y z.
I want to raise alerts, once cpu utilization exceeds "a" value for each of the nodes.

Below is the alert rule.

alert: cpu_utilization
expr: 100 - (avg by(instance) (irate(node_cpu_seconds_total{job="--",mode="idle"}[5m])) * 100) > a
labels:
  severity: critical
annotations:
  summary: CPU utilization has crossed a%


QUERY: 
1) How can I use the same rule for multiple nodes/jobs? Is there any way to update job names dynamically? As I want to avoid multiple alert rules for each job. 


Thanks,

Stuart Clark

unread,
Apr 17, 2021, 6:10:06 AM4/17/21
to akshay sharma, Prometheus Users
Just remove the job selector from your query and it will alert for all jobs
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Reply all
Reply to author
Forward
0 new messages