Take snapshot of Prometheus data and import it into another Prometheus server

[Muzaffar Mohammed]

Hello All,

Please, I have been looking for a tool to export subset of Prometheus data (e.g. 2 months of data out of 1 year data) and inject it into another Prometheus server. I looked in the web, but could not find such a tool (or capability in Prometheus). 

I checked the HTTP API query, but there is no way to inject the retrieved data into another Prometheus server. Also the remote_read is not an option as the two servers are in different networks (that could not be linked together).

Do you know about such a tool please?

Thanks in advance,

Best regards,

Muzaffar

[GANESH VERNEKAR]

Hi Muzaffar,

You can take the blocks from the data dir of one Prometheus and place it in another Prometheus while enabling vertical compaction. That way the 2nd Prometheus will pick up the blocks from the other Prometheus.

You can pick the blocks which satisfy your time ranges, else there is no other way to snapshot a range of data yet.

Thanks,

Ganesh

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bc16733d-38e5-4ceb-8349-7fa76e639030%40googlegroups.com.

[Muzaffar Mohammed]

Hi Ganesh,

Thanks a lot for your quick feedback!

I will test your suggestion. One point please, do you have a link on how to enable Vertical Compaction please?

Thank you !

Best regards,

Muzaffar

On Monday, February 24, 2020 at 3:10:29 PM UTC+1, GANESH VERNEKAR wrote:

Hi Muzaffar,

You can take the blocks from the data dir of one Prometheus and place it in another Prometheus while enabling vertical compaction. That way the 2nd Prometheus will pick up the blocks from the other Prometheus.

You can pick the blocks which satisfy your time ranges, else there is no other way to snapshot a range of data yet.

Thanks,

Ganesh

On Mon, Feb 24, 2020 at 7:32 PM Muzaffar Mohammed <muzaffar...@pix4d.com> wrote:

Hello All,

Please, I have been looking for a tool to export subset of Prometheus data (e.g. 2 months of data out of 1 year data) and inject it into another Prometheus server. I looked in the web, but could not find such a tool (or capability in Prometheus). 

I checked the HTTP API query, but there is no way to inject the retrieved data into another Prometheus server. Also the remote_read is not an option as the two servers are in different networks (that could not be linked together).

Do you know about such a tool please?

Thanks in advance,

Best regards,

Muzaffar

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.

[Muzaffar Mohammed]

Can I assume this CLI option enables vertical compaction please?

--storage.tsdb.allow-overlapping-blocks

It is mentioned as EXPERIMENTAL

Thanks,

Best regards,

Muzaffar

[GANESH VERNEKAR]

On Mon, Feb 24, 2020 at 8:03 PM Muzaffar Mohammed <muzaffar...@pix4d.com> wrote:

Can I assume this CLI option enables vertical compaction please?

--storage.tsdb.allow-overlapping-blocks

Yes this is the flag. You can enable once till the blocks are not overlapping anymore (happens after compaction), and then disable the flag after that.

If you are putting the blocks into a fresh Prometheus server, you won't require that flag at all as there won't be any overlapping blocks.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bc16733d-38e5-4ceb-8349-7fa76e639030%40googlegroups.com.

--

You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/aed699ab-9d8d-4d3f-81bb-212b811c8188%40googlegroups.com.