Unknown series references

[alexb...@gmail.com]

Prometheus version: v2.15.2

Problem：the size of  prometheus's wal is 42G,which will lead to OOM,and the prometheus can't restart,unless to remove the wal manually.

My question is:

1、Whether the  "Unknown series references" error is the reason of causing the wal too big?

1、Under what circumstances will "Unknown series references" happen? 

2、What is the  reason that lead to   "Unknown series references"  ? and how can I avoid it?

Many thanks!

[Matthias Rampke]

Prometheus 2.15 is rather old, have you considered upgrading to the latest version? There have been many fixes around the WAL and crash handling since then.

/MR

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/32fdd8b0-9bef-4899-abc3-d8c0006ef1fen%40googlegroups.com.

[alexb...@gmail.com]

Thanks! Yes，We are going to upgrade it.but you know we have too many environments,and we can't upgrade all these environments because it involves a lot of other things