Breaking down of one prometheus.yml file?

[pratyush ranjan]

I am using Prometheus for our monitoring and I have a lot of configs (our prometheus.yml main config file is 8000+ lines long).

I would like to divide this out into logical groupings so that it becomes much readable. I came to know that Prometheus doesn't support this and we can use configuration management systems like Ansible.

Has anyone done this with their Prometheus config file? If so, how did you do it?

[Christian Hoffmann]

Hi,

Yes, our Prometheus configs are auto-generated via Puppet, based on
templates. I'm afraid I can't share the code and it wouldn't make that
much sense anyway, as this part is highly customized for our environment.

What parts of the config make your file that long? Is it some
static_config part for service discovery?
If this is the case, you could easily switch to file_sd to move the
target lists to their own files.

Kind regards,
Christian

[pratyush ranjan]

Hi Christian,

Our Prometheus configs are also automated via Ansible, based on template. But in the end, all the lines are inserted to prometheus.yml file after 'scrape_configs'.We have a lot of AWS instances and each project using multiple exporter/services that's why the file is too long.

Yeah, I know about file_sd but it can hardly convert 8000+ lines file into 5000+ lines. I want to breakdown a single file of prometheus.yml into multiple like depending on projects and then using include(or similar) in main Prometheus config file (I can't seem to find any documentation like include in Prometheus). So, how can we do that?

Thanks

[Ben Kochie]

There are other advantages to moving targets from static_configs to file_sd_configs.

With targets in file_sd_configs, Prometheus knows how to reload these files without reloading all targets. This improves the scheduling of scrapes, as when you reload Prometheus, all in-flight scrapes are canceled.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/3fb33998-9ebf-4a14-9437-ea788f0400b7o%40googlegroups.com.

[Brian Brazil]

On Tue, 16 Jun 2020 at 11:06, Ben Kochie <sup...@gmail.com> wrote:

There are other advantages to moving targets from static_configs to file_sd_configs.

With targets in file_sd_configs, Prometheus knows how to reload these files without reloading all targets. This improves the scheduling of scrapes,

 

as when you reload Prometheus, all in-flight scrapes are canceled.

The only time we cancel in-flight scrapes is at shutdown.

Brian

 

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmph5C%3DTYfCwGqZ34fbozPYk5Uzwhvfej6LGWxu0qWj%2BrQ%40mail.gmail.com.

--

Brian Brazil

www.robustperception.io

[pratyush ranjan]

Hi,

Our Prometheus configs are also automated via Ansible, based on template. But in the end, all the lines are inserted to prometheus.yml file after 'scrape_configs'.We have a lot of AWS instances and each project using multiple exporter/services that's why the file is too long.

Yeah, I know about file_sd but it can hardly convert 8000+ lines file into 5000+ lines. I want to breakdown a single file of prometheus.yml into multiple like depending on projects and then using include(or similar) in main Prometheus config file (I can't seem to find any documentation like include in Prometheus). So, how can we do that?

Thanks