Prometheus HA with Influx Remote storage
567 views
Skip to first unread message
neha...@gmail.com
Mar 4, 2018, 10:49:03 PM3/4/18
to Prometheus Users
Hi Guys,
I am trying to design HA with Prometheus (scraper) and influxDB (remote storage living in different DC). I believe If I am running two identical Prometheus servers (HA mode) both having remote-write configured to influxDB (or Influx-relay) I will end up sending duplicate data over network for long term storage. How can I dedupe metrics from both Prometheus server before sending it to influxdb?
Thanks
Brian Brazil
Mar 5, 2018, 3:27:52 AM3/5/18
to Nehal Syed, Prometheus Users
On 5 March 2018 at 03:49, <neha...@gmail.com> wrote:
Hi Guys,I am trying to design HA with Prometheus (scraper) and influxDB (remote storage living in different DC). I believe If I am running two identical Prometheus servers (HA mode) both having remote-write configured to influxDB (or Influx-relay) I will end up sending duplicate data over network for long term storage. How can I dedupe metrics from both Prometheus server before sending it to influxdb?
You can't. The easiest way is to have only one Prometheus send to the remote storage cluster.
Brian Brazil
neha...@gmail.com
Mar 13, 2018, 7:00:58 PM3/13/18
to Prometheus Users
HI Brian,
Thanks for your answer. Can you please tell me if prometheus remote write api support writing to influxdb-relay service instead of influxdb directly?
Thanks
Brian Brazil
Mar 14, 2018, 3:41:28 AM3/14/18
to Nehal Syed, Prometheus Users
On 13 March 2018 at 23:00, <neha...@gmail.com> wrote:
HI Brian,Thanks for your answer. Can you please tell me if prometheus remote write api support writing to influxdb-relay service instead of influxdb directly?
That's a question for Influx, but given that code hasn't been touched in two years I'd say no. You'd need an adapter.
Brian
Thanks
On Monday, March 5, 2018 at 12:27:52 AM UTC-8, Brian Brazil wrote:On 5 March 2018 at 03:49, <neha...@gmail.com> wrote:Hi Guys,I am trying to design HA with Prometheus (scraper) and influxDB (remote storage living in different DC). I believe If I am running two identical Prometheus servers (HA mode) both having remote-write configured to influxDB (or Influx-relay) I will end up sending duplicate data over network for long term storage. How can I dedupe metrics from both Prometheus server before sending it to influxdb?You can't. The easiest way is to have only one Prometheus send to the remote storage cluster.--Brian Brazil
--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/11b0fdd1-a892-4a62-9a2d-b8b521e38b70%40googlegroups.com.
Brian Brazil
thiern...@polyconseil.fr
Mar 14, 2018, 6:16:06 AM3/14/18
to Prometheus Users
How are you dealing for InfluxDB's HA? If you have 2 InfluxDB, can't you just dedicate one InfluxDB for one prometheus server?
Brian Brazil
Mar 14, 2018, 6:25:51 AM3/14/18
to thiern...@polyconseil.fr, Prometheus Users
On 14 March 2018 at 10:16, <thiern...@polyconseil.fr> wrote:
How are you dealing for InfluxDB's HA? If you have 2 InfluxDB, can't you just dedicate one InfluxDB for one prometheus server?
There's not any point in using a single InfluxDB, remote write only makes sense for an InfluxDB cluster as otherwise you're still limiting yourself to the storage that fits on one machine.
Brian
Le lundi 5 mars 2018 04:49:03 UTC+1, neha...@gmail.com a écrit :Hi Guys,I am trying to design HA with Prometheus (scraper) and influxDB (remote storage living in different DC). I believe If I am running two identical Prometheus servers (HA mode) both having remote-write configured to influxDB (or Influx-relay) I will end up sending duplicate data over network for long term storage. How can I dedupe metrics from both Prometheus server before sending it to influxdb?Thanks
--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/392928c9-7cc3-4467-9b13-d346eb519ddf%40googlegroups.com.
thiern...@polyconseil.fr
Mar 14, 2018, 6:40:59 AM3/14/18
to Prometheus Users
My point is to deal with HA, you must have both prometheus and influxDB in HA.
InfluxDB cluster is probably the way to go, but is available only with entreprise version. So the other way to achieve HA, is to 2 pipelines and storage with the same data.
BTW, I'm totally agree with you with the limitation of the storage size (It should fits on one machine).
Le mercredi 14 mars 2018 11:25:51 UTC+1, Brian Brazil a écrit :
On 14 March 2018 at 10:16, <thiern...@polyconseil.fr> wrote:How are you dealing for InfluxDB's HA? If you have 2 InfluxDB, can't you just dedicate one InfluxDB for one prometheus server?There's not any point in using a single InfluxDB, remote write only makes sense for an InfluxDB cluster as otherwise you're still limiting yourself to the storage that fits on one machine.Brian
Le lundi 5 mars 2018 04:49:03 UTC+1, neha...@gmail.com a écrit :Hi Guys,I am trying to design HA with Prometheus (scraper) and influxDB (remote storage living in different DC). I believe If I am running two identical Prometheus servers (HA mode) both having remote-write configured to influxDB (or Influx-relay) I will end up sending duplicate data over network for long term storage. How can I dedupe metrics from both Prometheus server before sending it to influxdb?Thanks
--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/392928c9-7cc3-4467-9b13-d346eb519ddf%40googlegroups.com.
Nehal Syed
Mar 14, 2018, 2:17:44 PM3/14/18
to thiern...@polyconseil.fr, Prometheus Users
Thanks guys for quick response, I have two Prometheus (HA mode) running and I wanted to send out data to influxdb-relay (2 instance running behind LB) which will replicate same data to 2 Influxdb instance (HA/non-enterprise). For now the only HA configuration I can have is that each Prometheus post data to corresponding Influxdb and In this case data will be slightly different on each influxdb.
Regards
Nehal
--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/Ak2z7mfvZzw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/bf572988-b846-4857-a5f8-0d5a4b53a8cd%40googlegroups.com.
Martin Chodúr
Mar 14, 2018, 10:19:10 PM3/14/18
to Prometheus Users
Actually you could try using InfluxDB native read and write support and duplicate the data by sending them to Telegraf which would send them to two influx instances.
I wanted to try this for ha long term metrics persistance but had no time so this is just more of a vision than prooved solution but i thing it might work actually.
Still there vould be problem with deduping the data.
Reply all
Reply to author
Forward
0 new messages