Building multi-target exporter in Python

[Elliott Balsley]

I’d like to build a multi-target exporter, similar to the blackbox exporter, where it takes an IP address as part of the URL path and uses that to collect metrics from hardware devices. This would be my first time writing a Prometheus exporter. From the Python client examples, I’ve created a CustomCollector class which collects only when scraped, running with start_http_server. But I can’t figure out how to access the URL path as a variable. Is that possible with this method or do I need to use a different web server like Flask or WSGI? A very simple example is below.

from prometheus_client.core import GaugeMetricFamily, REGISTRY
from prometheus_client.registry import Collector
from prometheus_client import start_http_server
import random
import time

class CustomCollector(Collector):
    def collect(self):
        """Yield metrics only when scraped"""

        metrics = self.read_from_device()
        for metric in metrics:
            yield GaugeMetricFamily(metric['name'], 'help text', value=metric['value'])

    def read_from_device(self):
        """Blocking code that reads and parses data from hardware"""

        return [
            {'name': 'request_processing_seconds', 'value': random.random()},
        ]

REGISTRY.register(CustomCollector())


if __name__ == '__main__':
    start_http_server(port=8000)
    while True:
        time.sleep(86400)

[Brian Candler]

The way that blackbox_exporter itself handles this (in Go) is to create a fresh disposable registry for every request, inside the http handler:

https://github.com/prometheus/blackbox_exporter/blob/master/prober/handler.go#L125-L148

and then each probe request registers its metrics onto that registry, e.g.

https://github.com/prometheus/blackbox_exporter/blob/master/prober/icmp.go#L84-L88

I think this ought to be possible in Python too. I guess you would need to call make_wsgi_app at the end of your handler, and then invoke it. [Untested]