Multiple Prometheus monitor the same targets, and each node's Prometheus will call the remote write, resulting in data duplication

ronghao li

unread,

Jun 28, 2021, 3:07:44 AM6/28/21

to Prometheus Users

Multiple Prometheus monitor the same targets, and each node's Prometheus will call the remote write, resulting in data duplication.How to solve this problem？

Stuart Clark

unread,

Jun 28, 2021, 3:41:06 AM6/28/21

to ronghao li, Prometheus Users

Deduplication is handled by the remote write destination, such as Thanos. Usually there is the requirement to add an external label identifying the cluster and the replica to allow this to happen.

On 28 June 2021 08:07:44 BST, ronghao li <foreverlo...@gmail.com> wrote:

Multiple Prometheus monitor the same targets, and each node's Prometheus will call the remote write, resulting in data duplication.How to solve this problem？

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

sayf eddine Hammemi

unread,

Jun 28, 2021, 3:58:06 AM6/28/21

to Stuart Clark, ronghao li, Prometheus Users

Hello, I discovered this a while ago, https://github.com/beorn-/highlander, it is a reverse proxy for remote_write that deduplicates from multiple prometheis. this ofc in the case of the remote destination does not support dedup out of the box or is expensive resource-wise

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/7A738BD7-D4EC-44A0-9343-BF4520A069C9%40Jahingo.com.

Reply all

Reply to author

Forward