Is this valid that all my custom metrics has the same job?

[stav alfi]

Intro

I noticed that all of my custom metrics has the same job value: prometheus-io-scrape.

as you can see in one of my metrics:

Questions

• How do I change it to the name of the service? Should I manually add a label job to every metric in the codebase?

• Why I'm seeing prometheus-io-scrape job everywhere?

More Info

I'm using prom-client for node.js, and Rancher is our prometheus configuration manager

[Brian Candler]

That's the "job" label.  It's added automatically by prometheus, and is set to the name of the scrape job which collected the metric, as documented here:

https://prometheus.io/docs/concepts/jobs_instances/

You should leave it alone.  It's a true piece of information (i.e. the name of the actual scrape job which collected the data), and together with the instance label, it helps ensure that all metrics have unique sets of labels.

However, you *can* customize the instance label - prometheus only adds the default address:port label if you haven't set it to anything else explicitly.  See:

https://www.robustperception.io/controlling-the-instance-label

You have to ensure that whatever value you chose is sufficiently unique, i.e. all timeseries for the same metric have different instance labels.