TSDB Index Format Rollback from Prometheus v3 to Pre-v2.55.0

[Gavine]

Hi Prometheus Dev Group,

As we know, Prometheus v3 will introduce a breaking change to TSDB, specifically the index format V3, with v2.55.0 serving as the migration gateway.

Our product has a requirement to support rollback from v3 to versions prior to v2.55.0. Could we have your opinion on this? Specifically, do you foresee any issues if we develop a conversion tool to change the version number in the index files back to 2 before a rollback?

From what we understand, Prometheus v2 (starting from certain older versions) has already stopped reading/parsing the fields removed in format V3, i.e. Label Index 1...N and the Label Index Table, correct? By reviewing the related changes in v2.55.0, it seems the modifications were primarily to make format V3 recognizable while applying the same logic as format V2.

Is there anything else we should consider for such a rollback mechanism?

Thanks in advance for your insights!

Best regards,

Gavine

[Bartłomiej Płotka]

Hi!

I'm not familiar with exact reasons for rollback break in this case and if it's possible to e.g. backport to the last LTS.

However, we discussed formal procedure and process for storage changes here: https://github.com/prometheus/proposals/pull/40 - give us feedback on this!

This is for WAL but I might extend this to TSDB blocks. TL;DR is to support rollback to at least one LTS version back.

Kind Regards,

Bartek Plotka @bwplotka