Best practices for skipping unwanted metrics in Multiprocess mode

[amar]

Hello All,

We are running prometheus in Multiprocess mode for some python services and using push gateway for some services.

Here is sample code:

Service 1: 

Filename: test_prom_multiproc.py

 from prometheus_client import Counter, start_http_server, generate_latest, CollectorRegistry, multiprocess

import test_prom_counter

registry = CollectorRegistry()

multiprocess.MultiProcessCollector(registry)

total_req = Counter("total_req", "total req", ["api", "code"])

total_req.labels("api", "code").inc()

print(generate_latest(registry))

Service 2:
Filename: test_prom_counter.py

from prometheus_client import CollectorRegistry, Counter, start_http_server

registry = CollectorRegistry()

total_requests = Counter("total_requests", "total requests", ["status", "method"], registry=registry)

total_requests.labels("status","method").inc()

We are running a Monolith application and due to some dependencies we are ending up in the above situation.

As we are interested only in the Service1, when we try to scrape the metrics, we get metrics from Service1 and Service2. Since we have set the PROMETHEUS_MULTIPROC_DIR, the Service2 metrics are getting dumped to it along with Service1. 

We can have a hack or a smiple check to bypass the metrics from Service2 getting dumped to PROMETHEUS_MULTIPROC_DIR but we would like to hear if anyone has encountered this issue and can suggest any best practices.

Appreciate your help.

Thanks,

Amar