Prometheus Python Client - How to collect only on scrape and control targets?

[ee1]

Happy New Year.

The Python client examples I've been able to find all, including the one here,  start the HTTP server on the side, and run the metric-generating function in a loop regardless of whether, or at what frequency, prometheus is scraping that metric.

1 - With this client, is it possible, and is anyone aware of a simple example, of only executing the metric-generating function when a scrape comes in from prometheus,?

I notice that the HTTP server included ignores the URI of the request and returns the metrics regardless.  e.g. This works just fine: http://localhost:8000/foo?doesntmatter=anything

2 - Similarly is it possible/available Python example, of using the included HTTP server but parsing HTTP parameters that come in from prometheus in order to write a multi-target exporter like described here? 

Ultimate goal is to be able to write a blackbox-style exporter for anything where a metric can be pulled by some Python code.  e.g. Obscure appliances/metrics for which there is no easier answer like SNMP or existing Blackbox modules.

Thanks.

[Brian Candler]

On Sunday, 2 January 2022 at 03:05:16 UTC ee1 wrote:

1 - With this client, is it possible, and is anyone aware of a simple example, of only executing the metric-generating function when a scrape comes in from prometheus,?

Yes: for a gauge, you can register a callback function.  The function will be triggered on a scrape.

https://github.com/prometheus/client_python#gauge

Another way is using Custom Collectors:

https://github.com/prometheus/client_python#custom-collectors

(Obviously for counters, you need to increment or accumulate the counter whenever the metric-generating event happens - otherwise you wouldn't know the total value when the scrape occurs)

 

I notice that the HTTP server included ignores the URI of the request and returns the metrics regardless.  e.g. This works just fine: http://localhost:8000/foo?doesntmatter=anything

2 - Similarly is it possible/available Python example, of using the included HTTP server but parsing HTTP parameters that come in from prometheus in order to write a multi-target exporter like described here? 

Bolt it onto your own http.server instead. See the docs here: https://github.com/prometheus/client_python#http

"To add Prometheus exposition to an existing HTTP server, see the MetricsHandler class which provides a BaseHTTPRequestHandler. It also serves as a simple example of how to write a custom endpoint."

 

[ee1]

Thank you sir for that guidance.  Tested the callback function approach and that seems to be simple and fits the use case well.  This seems to work:

from prometheus_client import start_http_server, Gauge
import random
import time

g = Gauge('some_test_metric', 'TEST METRIC')

def test_gauge():
  x = random.random()
  print(x)
  return(x)

g.set_function(lambda: test_gauge())

if __name__ == '__main__':
    # Start up the server to expose the metrics.
    start_http_server(8000)

    while True:
      time.sleep(5)

 

Will have to do some more head scratching on bolting it on to own HTTP server (due to my own noobness in this area) but thanks for pointing out the direction.