Hi. I am using Prometheus version 2.26.0. The Prometheus has currently around only 8 lakh time series(0.8 Million), but the memory usage of Prometheus is touching as high as 76GB.
I have another Prometheus v 2.22.0 setup that is scraping different targets but is currently handling around 10 Million time series and that uses 52-55 GB of memory. I am not seeing anything abnormal in the logs of 2.26.0. Memory chunks are also usually at 5 Million in 2.26.0 whereas 2.22.0 usually have 55 Million.
How do I debug why is the memory usage abnormally high? Can someone please point me in the right direction?
Thanks in advance!