Downsampling data to a secondary Prometheus server?

[Nils Lagerkvist]

I'm looking at the possibility to create an exporter for Prometheus data to accomplish downsampling. Then let a secondary Prometheus instance pull that data and store the downsampled data with a long retention period. 

First, is there already a exporter for prometheus data?

Secondly, is the best way to do this aggregation with the HTTP API? or is there a better way to do it?

Lastly, is there a better way to accomplish this (maybe I should have started with this question)?

[Matthias Rampke]

What you are looking for is built into Prometheus as "federation":

https://prometheus.io/docs/operating/federation/

It is a special way of scraping.

The secondary prometheus can have an independent scrape cycle. You can
also aggregate in the primary one using rules and then only select
those rules for federation, thereby reducing the number of timeseries
to store.

/MR

> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-devel...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Matthias Rampke
Engineer

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany | +49
173 6395215

Managing Director: Alexander Ljung | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG Charlottenburg |
HRB 110657B

[Nils Lagerkvist]

Thanks for the quick answer, I will look into this feature.

//Nils

On Thursday, 4 February 2016 17:05:09 UTC+1, Matthias Rampke wrote:

What you are looking for is built into Prometheus as "federation":

https://prometheus.io/docs/operating/federation/

It is a special way of scraping.

The secondary prometheus can have an independent scrape cycle. You can
also aggregate in the primary one using rules and then only select
those rules for federation, thereby reducing the number of timeseries
to store.

/MR

On Thu, Feb 4, 2016 at 4:02 PM, Nils Lagerkvist
<nils.la...@gmail.com> wrote:
> I'm looking at the possibility to create an exporter for Prometheus data to
> accomplish downsampling. Then let a secondary Prometheus instance pull that
> data and store the downsampled data with a long retention period.
>
> First, is there already a exporter for prometheus data?
> Secondly, is the best way to do this aggregation with the HTTP API? or is
> there a better way to do it?
>
> Lastly, is there a better way to accomplish this (maybe I should have
> started with this question)?
>



--