Multiprocessing of python_client for independent python deamons

[Mounya A]

Hii everyone, 
         We have multiple independent deamons of same application in a machine. I would like to serve aggregated metrics from a single port , rather than starting server in each deamon and treating them as a separate time series. I can handle the exposition part with try.. except so that the first deamon that gets hold of the port  can start the server. My doubt is regarding the correctness of metrics data if i use the python client's multiprocessing support. (as it is not a direct use of gunicorn or multiprocessing module) . Is there any chance of ending up with incorrect data if i use this approach . Thanks in advance.

[Brian Brazil]

On Fri, 28 Aug 2020 at 07:39, Mounya A <alli.mo...@gmail.com> wrote:

Hii everyone, 
         We have multiple independent deamons of same application in a machine. I would like to serve aggregated metrics from a single port , rather than starting server in each deamon and treating them as a separate time series. I can handle the exposition part with try.. except so that the first deamon that gets hold of the port  can start the server. My doubt is regarding the correctness of metrics data if i use the python client's multiprocessing support. (as it is not a direct use of gunicorn or multiprocessing module) . Is there any chance of ending up with incorrect data if i use this approach . Thanks in advance. 

If they're independent daemons then each should have their own /metrics. The multiprocessing mode comes with a number of disadvantages, and should only be used if you've no other option.

--

Brian Brazil

www.robustperception.io

[Mounya A]

    Thanks for the reply Brian. As far as I know, most of the disadvantages are caused by lot of process churn, ours are 3-5 long lived processes and  we need only simple metrics like Counter and Summary. Still ,will we be affected by those disadvantages? Are there any limitations other than those mentioned in the documentation??Thanks in advance.

[Brian Brazil]

On Fri, 28 Aug 2020 at 11:45, Mounya A <alli.mo...@gmail.com> wrote:

    Thanks for the reply Brian. As far as I know, most of the disadvantages are caused by lot of process churn, ours are 3-5 long lived processes and  we need only simple metrics like Counter and Summary. Still ,will we be affected by those disadvantages?

Yes, the disadvantages are due to the implementation. If you've known long running processes, it'd be better to scrape them directly.

 

Are there any limitations other than those mentioned in the documentation??Thanks in advance.

The list should be complete. 

 Brian

On Friday, August 28, 2020 at 12:14:30 PM UTC+5:30 brian....@robustperception.io wrote:

On Fri, 28 Aug 2020 at 07:39, Mounya A <alli.mo...@gmail.com> wrote:

Hii everyone, 
         We have multiple independent deamons of same application in a machine. I would like to serve aggregated metrics from a single port , rather than starting server in each deamon and treating them as a separate time series. I can handle the exposition part with try.. except so that the first deamon that gets hold of the port  can start the server. My doubt is regarding the correctness of metrics data if i use the python client's multiprocessing support. (as it is not a direct use of gunicorn or multiprocessing module) . Is there any chance of ending up with incorrect data if i use this approach . Thanks in advance. 

If they're independent daemons then each should have their own /metrics. The multiprocessing mode comes with a number of disadvantages, and should only be used if you've no other option.

 

--

Brian Brazil

www.robustperception.io

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/b1724855-ac9d-4f47-a7d4-35440eeddae5n%40googlegroups.com.

--

Brian Brazil

www.robustperception.io