Prometheus startup fails

[AdP]

Hello,

I am facing an issue with Prometheus where my set up fails to start, and exits due to OOM, even before all the WAL segments are completely loaded.

This is being run on AWS ECS and the prometheus task is allocated 15GB of memory, no hard limit.

When I check the disk usage of each of the folders created by Prometheus, I find the following:

 W

The WAL size appears to remain 3.9G during the entirety of prometheus WAL reload, while the chunks_head keeps increasing, and ultimately the prometheus container dies down because of OOM. This happens repeatedly.

I am puzzled about what causes the issue.

Please note the version of prometheus in the screenshot.