Serving complex alert rule directory structure to Prometheus

307 views
Skip to first unread message

arnold.b...@googlemail.com

unread,
Mar 14, 2018, 5:46:25 AM3/14/18
to Prometheus Users
Hi,

I have a complex directory structure (built by https://github.com/inovex/trovilo) container alert files that is automatically (re-)generated. Prometheus accepts actual file paths (golang file globs) on "rule_files" in the configuration only.

I think that might be a feature request worth since I don't see how to do this now. See also https://github.com/prometheus/prometheus/issues/3954 (closed).

Any ideas?


$ tree prometheus-alerts
/etc/prometheus-alerts
|-- team1/
|   `-- configmap1/
|       |-- alert1.yaml
|       `-- alert2.yaml
`-- team2/
    |-- configmap1/
    |   |-- alert1.yaml
    |   |-- alert2.yaml
    |   `-- alert3.yaml
    `-- configmap2/

Arnold

Brian Brazil

unread,
Mar 14, 2018, 5:51:06 AM3/14/18
to arnold.b...@googlemail.com, Prometheus Users
On 14 March 2018 at 09:46, arnold.bechtoldt via Prometheus Users <promethe...@googlegroups.com> wrote:
Hi,

I have a complex directory structure (built by https://github.com/inovex/trovilo) container alert files that is automatically (re-)generated. Prometheus accepts actual file paths (golang file globs) on "rule_files" in the configuration only.

I think that might be a feature request worth since I don't see how to do this now. See also https://github.com/prometheus/prometheus/issues/3954 (closed).

Any ideas?

You should either arrange your files into a known set of directories, or have your configuration management generate the list of all files/directories when templating out your prometheus.yml.

Brian
 


$ tree prometheus-alerts
/etc/prometheus-alerts
|-- team1/
|   `-- configmap1/
|       |-- alert1.yaml
|       `-- alert2.yaml
`-- team2/
    |-- configmap1/
    |   |-- alert1.yaml
    |   |-- alert2.yaml
    |   `-- alert3.yaml
    `-- configmap2/

Arnold

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/89a45b88-62b2-4755-86fe-5efc1be48fed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Message has been deleted

chro...@gmail.com

unread,
Mar 14, 2018, 6:56:18 AM3/14/18
to Prometheus Users
Hello, 

do you know why my last answer was deleted? Did I miss any rules?


Am Mittwoch, 14. März 2018 10:51:06 UTC+1 schrieb Brian Brazil:
On 14 March 2018 at 09:46, arnold.bechtoldt via Prometheus Users <promethe...@googlegroups.com> wrote:
Hi,

I have a complex directory structure (built by https://github.com/inovex/trovilo) container alert files that is automatically (re-)generated. Prometheus accepts actual file paths (golang file globs) on "rule_files" in the configuration only.

I think that might be a feature request worth since I don't see how to do this now. See also https://github.com/prometheus/prometheus/issues/3954 (closed).

Any ideas?

You should either arrange your files into a known set of directories, or have your configuration management generate the list of all files/directories when templating out your prometheus.yml.

Brian
 


$ tree prometheus-alerts
/etc/prometheus-alerts
|-- team1/
|   `-- configmap1/
|       |-- alert1.yaml
|       `-- alert2.yaml
`-- team2/
    |-- configmap1/
    |   |-- alert1.yaml
    |   |-- alert2.yaml
    |   `-- alert3.yaml
    `-- configmap2/

Arnold

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.



--

Brian Brazil

unread,
Mar 14, 2018, 6:58:24 AM3/14/18
to chro...@gmail.com, Prometheus Users
On 14 March 2018 at 10:56, <chro...@gmail.com> wrote:
Hello, 

do you know why my last answer was deleted? Did I miss any rules?

The spam filter has false positives from time to time, messages are usually cleared out within a day.

Brian
 
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/a289424f-035c-46ef-8f57-9270f68e79ab%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

Brian Brazil

unread,
Mar 14, 2018, 6:59:34 AM3/14/18
to chro...@gmail.com, Prometheus Users
On 14 March 2018 at 10:33, <chro...@gmail.com> wrote:
Hello, 

according to the prometheus source code the filepath.Glob() function is used. (See https://github.com/prometheus/prometheus/blob/master/cmd/prometheus/main.go#L326-L339) This functions should support that structure.  IMHO the regex https://github.com/prometheus/prometheus/blob/master/config/config.go#L33 is too restrictive. 

This would be inconsistent with file_sd_configs where we are doing more complex things than a simple glob, and having two different ways in which globs work would cause confusion.

Brian
 


I constructed a small test.

content test.go:
package main

import (
"path/filepath"
"fmt"
)

func main() {
   files, err := filepath.Glob("./*/*/*.yaml")
   if err != nil {
   fmt.Println(err)
   }
   for _, file := range files {
   fmt.Println("Found file: ", file) 
   }
}

TestScript:

$ mkdir -p team1/configmap1 team2/configmap2
$ touch team1/configmap1/config.yaml
$ touch team2/configmap2/config.yaml
$ go run test.go
Found file:  team1/configmap1/config.yaml
Found file:  team2/configmap2/config.yaml


--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Reply all
Reply to author
Forward
0 new messages