Label injection at exposition time (Python library)

[Pau Freixes]

HI to all,

Brian pointed to me to that link [1] which basically claims that in Prometheus - compared to other TimeSeries databases - there is no necessity of prefixing the names of the metrics. And TBH I like the idea of not having to prefix the metric name with for example the service name, I can see some potential use cases on that, for example being able to gather all metrics seen from upstream servcies when they are calling your service, by prefixing the metrics it would become a painful process.

But having the feeling that my original question is still applicable, but for labels. So let me rephrase the question that I've originally opened here [2].

Seems that Prometheus Python client prescribes a pattern for declaring the metrics at the module level, which has the good benefit of making it available quite straightforward by just importing that specific symbol, also allows the system to keep everything in memory - not deallocated by the GC - for the whole process time. And its there when the labels epxected for a specific metric are enumerated, for example:

REQUESTS = Counter(

    "starlette_requests_total", "Total count of requests by method and path.", ["method", "path_template"]

)

Later on in your code, you do something like this:

REQUESTS_IN_PROGRESS.labels(method=method, path_template=path_template).inc()

My questions is, how could we manage the necessity of having to add the same label value for all of the metrics that are generated within an application? By following the prescribed pattern we would be forced to pollute all of the metrics with something like this:

REQUESTS = Counter(

    "starlette_requests_total", "Total count of requests by method and path.", ["method", "path_template", "service_name"]

)

And later on in our code do something like this:

REQUESTS_IN_PROGRESS.labels(method=method, path_template=path_template, service_name=get_service_name()).inc()

I guess that if you have just a bunch of metrics and a few services might be manageable, but it quickly becomes a lot of repeated code that not even the developer should take care. Is there any alternative to do so in a generic and none friction way?

I'm wondering if we could create our own Registry - I do not know if `Resgistry` is a first citizen class that can be used by the developers - and use the same instance in all of our metrics and when the data is gathered by the `generate_latest` do something like this:

from myregistry import RegistryWithServiceName

if RegistryWithServiceName.get_service_name() is None:

    RegistryWithServiceName.set_service_name(get_service_name())

generate_latest(REGISTRY)

Behind the scenes, RegistryWithServiceName would need to update the collected metrics by adding a new label to all of them, so having as a result a list of metrics with their original labels plus the one that we inserted during the `generate_latest` execution

WDYT?

[1] https://www.robustperception.io/target-labels-not-metric-name-prefixes

[2] https://github.com/prometheus/client_python/issues/571

[Pau Freixes]

[Seems that is doable, Ive ended up doing something like this

```
class MyRegistry(CollectorRegistry)

def collect(self):
"""Yields metrics from the collectors in the registry."""
for metric in super().collect():
metric.samples = [
Sample(
name=sample.name,
labels=dict(**sample.labels,
service_name=self._service_name),
value=sample.value,
timestamp=sample.timestamp,
exemplar=sample.exemplar
) for sample in metric.samples ]
yield metric
```

But this implementation have some sort of coupling with the current
implementation, first needs to know what attributes the `Sample`
object has, which it's not clear to me if this object is a public
object or not, and second also relays on the fact that the creation of
a new list of samples would not have any downside effect behind the
scenes, so in any case, this would mutate the original metric. Seems
that nowadays this can be guaranteed, but who knows what could happen
in the future.

Would it make sense to move this feature within the Python client in some way?

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6f760079-348b-44d6-a98e-1f4329d52bf7o%40googlegroups.com.



--
--pau