metrics by process

[Laura Fernández Becerra]

Hi everyone

I would like to know if is it possible by using Prometheus to record the CPU, RAM, net I/O of every process in a system for an interval of time.

Thanks in advance!

[Christian Hoffmann]

Hi,

Something like that might be (almost) possible with a process_exporter
such as [1].

Keep in mind that a process list with all its stats will always be a
specific snapshot of some point in time, even with a very short scrape
interval. For example, short-lived processes might well be missed by
such a scrape, even if they take up a large part of CPU time. In many
cases its more useful to gather such statistics on some higher level
such as container, systemd or cgroup (which both techniques use).
cadvisor and maybe the systemd_exporter would be good candidates for that.

Also keep in might that depending on your workload you might see a large
number of processes with individual names/PIDs over time. This may lead
to cardinality explosions which are hard to handle for Prometheus.

In other words: If you accept some tradeoffs, it's possible, but if you
intend to leave this running for a larger fleet and/or a longer
timeframe, you might want to look into other tools as well.

If you need more specific insights, I would suggest looking for some way
of tracing (e.g. perf, kprobes, execsnoop) instead.

Kind regards,
Christian


[1] https://github.com/ncabatoff/process-exporter

[Laura Fernández Becerra]

Thanks Christian

I'm trying to complete this dataset with CPU, RAM and other system metrics that could be interesting. Most of the tools gives you a whole view of the system or containers or pods.. is there any tool that could provides me the cpu consumption for an interval of time? I think I could assign this value to all the processes that wouldbe running at that moment. Thanks again