Alert thresholds configuration via UI (Prometheus or Thanos Ruler) & Alert Manager

89 views
Skip to first unread message

Venkatesh Goud

unread,
Jun 24, 2021, 2:26:14 AM6/24/21
to Prometheus Users
we are looking into below problem statement , could any suggest options to achieve this.

  How to build the rules file automatically as per resource type thresholds if end user  
  provides the  resource, metric and interval to check.

Example: If we provide the   Metric - CPU 
                                                  Threshold - 90
                                                   interval - 10 mins for particular VM/product.
it should create rule in prometheus/thanos  rule file automatically.

This will hep the end user to not worry about the write the rules logic /code?

Same mechanism for alert manager - configurations for silence, group and external routes can be modified automatically with some interface ?

Any suggests /links/leads helps us alot 

Regards,
Venkat.

Ian Billett

unread,
Jun 24, 2021, 6:23:46 AM6/24/21
to Venkatesh Goud, Prometheus Users
Hey Venkatesh,

It sounds like what you are describing here is a templating problem: given a set of user-defined inputs, generate a well-formed output rules file. Templating is a very common taks, and there will be first-class tools available for it in your programming language of choice.

To the best of my knowledge, I'm not aware of any existing community tool that do exactly what you are looking for, but checkout https://promtools.dev/alerts/errors which does something quite similar but for SLO alerts.

Best,

Ian


--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/020ce107-ee97-4fb0-b680-60c0d46c8085n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages