Estimating memory usage

[ili...@luminatewireless.com]

Hi again,

I'd like to do a back-of-the-envelope calculation to determine how much data will fit in local storage. Let's say I'm scraping 1000 nodes, and each node exposes about 15000 bytes of data, which I'm saving every 10 seconds. I'm trying to get a sense of how much space would be needed to fit 1 weeks worth of data, 1 month, etc. I see that there are many ways to configure the storage, which is making it hard for me to calculate this. Short of just writing an exporter to simulate this and run an experiment accordingly, is there a way to roughly estimate how much space would be used?

Thanks,

Ilina

[Julius Volz]

What matters for disk storage space is mostly how many samples you ingest. To a minor extent, it also matters how many series you have, plus how many (and how long) labels they each have, but I'd focus on the number of total samples first.

You can derive the total number of ingested samples by multiplying your number of targets by the number of series each exposes, then multiply that by how many times they would get scraped in 1 week, 1 month, etc.

Then you can use the handy table in https://prometheus.io/docs/operating/storage/#chunk-encoding and multiply one of those "bytes per sample" values with your number of ingested samples to arrive at the rough overall number of bytes to expect.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/768bbb04-7867-47a1-b3ac-e8642b837b4d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.