Multi cluster HA

32 views

Skip to first unread message

unread,

May 28, 2021, 7:35:05 AM5/28/21

to Prometheus Users

Hi team

Do we have multi cluster HA mechanism for Prometheus?

unread,

Jun 9, 2021, 3:56:04 PM6/9/21

to Mohan Nagandlla, Prometheus Users

On 28.05.21 17:04, Mohan Nagandlla wrote:
> Hi team
> Do we have multi cluster HA mechanism for Prometheus?

General HA idea in Prometheus is very simple and robust:

https://prometheus.io/docs/introduction/faq/#can-prometheus-be-made-highly-available
https://www.robustperception.io/high-availability-prometheus-alerting-and-notification

Not sure what you mean by "multi-cluster".

You could (and should) run the various instances of your Alertmanager
cluster in different clusters, if you run in multiple clusters.

A Prometheus server is usually set up to monitor local targets. So you
would run both members of an HA pair in the same cluster. And then run
at least one pair in each cluster.

As an exception to local-only scraping, you could setup cross cluster
meta-monitoring, so you will notice if a cluster goes completely
blind. This is an example how to do it:
https://grafana.com/blog/2021/04/08/how-we-use-metamonitoring-prometheus-servers-to-monitor-all-other-prometheus-servers-at-grafana-labs/

--
Björn Rabenstein
[PGP-ID] 0x851C3DA17D748D03
[email] bjo...@rabenste.in

Reply all

Reply to author

Forward

0 new messages