Migrate old time series to new time series in prometheus database

[Amin Borjian]

Hi.

We are looking to rename some of the metrics name (metric X -> Y). However, we do not want lose old data that has already been collected and stored by Prometheus in TSDB. history of metric is important for us.

We are looking for the following way:
1) Change the name of the metric, as a result of which the metric will be saved with a new name from now on.
2) After the above change, change the name of the old metrics in the Prometheus database.

But we did not find a way for the second part. Is there no way for it in Promehteus? In this case, what should be done in our scenario?

Thank you for your help.

[Julius Volz]

Hi Amin,

There's no built-in way to do 2) in Prometheus, so either you would have to build new tooling (probably in Go) for this or maybe it could be done in a relatively inefficient way like this:

1) Use "promtool tsdb dump" to dump your old data into a file.

2) Change the formatting of that file to valid OpenMetrics (e.g. insert "# EOF" to terminate metric groups, etc., see for example "OpenMetrics Novelties" in https://sysdig.com/blog/openmetrics-is-prometheus-unbound/).

3) Delete the old TSDB block dirs in your Prometheus data directory.

4) Backfill the OpenMetrics file using "promtool tsdb create-blocks-from openmetrics" (see https://prometheus.io/docs/prometheus/latest/storage/#backfilling-from-openmetrics-format)

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/ff915f6f-d9ee-4dd8-a0c0-2cd721df8792n%40googlegroups.com.

--

Julius Volz

PromLabs - promlabs.com

[Julius Volz]

(and of course in step 2, you would also do whatever metric renamings you want to do in the dumped file)