Automate creation of alerting rules in Prometheus Server

[vikram yerneni]

Fellas,

Within the Prometheus server on EKS Cluster, I can edit the configmaps and can add the alerting rules or expose the configmap through a configuration management tool (like Ansible) and can push the changes.

But I am looking for a light weight way (like an API call) where we can create, edit and support the Alerting Rules in Prometheus.

Has anyone using an unknown or known option for this setup?

Cheers!

[Felippe Bueno]

I don't think that's the way to do it.

I mean, you can handle that for sure.

But going with prometheus-operator is way easier.

https://github.com/helm/charts/tree/master/stable/prometheus-operator

Then if you still need custom rules just check 

additionalPrometheusRulesMap

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f4998fea-a040-4962-9585-756ccc99cc39o%40googlegroups.com.

[vikram yerneni]

Oh wow... ok ok, I never used this prometheus-operator before. I will set it up.

Thanks Felippe. 

On Tuesday, June 23, 2020 at 5:56:20 PM UTC-5, Felippe Bueno wrote:

I don't think that's the way to do it.

I mean, you can handle that for sure.

But going with prometheus-operator is way easier.

https://github.com/helm/charts/tree/master/stable/prometheus-operator

Then if you still need custom rules just check 

additionalPrometheusRulesMap

On Tue, Jun 23, 2020 at 7:21 PM vikram yerneni <vikr...@gmail.com> wrote:

Fellas,

Within the Prometheus server on EKS Cluster, I can edit the configmaps and can add the alerting rules or expose the configmap through a configuration management tool (like Ansible) and can push the changes.

But I am looking for a light weight way (like an API call) where we can create, edit and support the Alerting Rules in Prometheus.

Has anyone using an unknown or known option for this setup?

Cheers!

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.

[Felippe Bueno]

Yeah. It's a game-changer.

Let us know how it goes.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/82567bce-7388-4be8-ad63-afc91d2bf7abo%40googlegroups.com.

[vikram yerneni]

Will do.

Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/82567bce-7388-4be8-ad63-afc91d2bf7abo%40googlegroups.com.

[vikram yerneni]

I hit a road-block here...

Probably I should have mentioned it earlier, but here It is:  I already got Prometheus Server and associated Alertmanager instances up and running. I just need a way to manage the alerts creation. 

If I use this prometheus-operator route, this will need its own instance of Prometheus server and Alertmanager along with it, which does not work for us.

Is there a way to integrate the prometheus-operator setup with existing Prometheus Server setups?

Please do let me know.

Thanks

[Felippe Bueno]

You could use prometheus federation for scraping data from prometheus operations to your existing server.

However, that won't help creating rules. I mean, rule will take effect only to the prometheus operator.

I can't tell if there's a way out of the box to create this.

in my case, what isn't inside prometheus operator I manage using ansible+jenkins

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/609a7244-2d12-4b85-b656-687ddb8c2355o%40googlegroups.com.

[vikram yerneni]

Yes, looks like this prometheus-operator route is little complicated and the only way here for me is to use the configuration management utility to mess with the configmaps. 

I might go with the terraform/stash method of deploying the configmap changes and will push it through bamboo. 

Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/609a7244-2d12-4b85-b656-687ddb8c2355o%40googlegroups.com.