How to run pod based on prometheus alert

28 views
Skip to first unread message

rajendar t

unread,
Nov 16, 2021, 7:30:04 AM11/16/21
to promethe...@googlegroups.com
Is there any way we can run pod based on the alert fired from Prometheus , we have a scenario where we need to execute a pod based on the disk pressure threshold. I am able to create alert , but I need to execute a pod .How can I achieve that
groups:
  - name: node_memory_MemAvailable_percent
    rules:
    - alert: node_memory_MemAvailable_percent_alert
      annotations:
        description: Memory on node {{ $labels.instance }} currently at {{ $value }}% 
          is under pressure
        summary: Memory usage is under pressure, system may become unstable.
      expr: |
        100 - ((node_memory_MemAvailable_bytes{job="node-exporter"} * 100) / node_memory_MemTotal_bytes{job="node-exporter"}) > 80
      for: 2m
      labels:
        severity: warning

Thanks 
Rajendar 

Stuart Clark

unread,
Nov 16, 2021, 7:55:53 AM11/16/21
to rajendar t, promethe...@googlegroups.com
You can use a webhook to do anything you want. Just create a small app
(using whatever language you wish) that listens for webhook
notifications from Alertmanager than then executes your code when
triggered. You can then set your alert routing to send the notification
for the alert(s) you wish.

--
Stuart Clark
Reply all
Reply to author
Forward
0 new messages