Go client pushgateway delete by instance

40 views
Skip to first unread message

Aaron Reisman

unread,
Apr 18, 2023, 9:36:42 AM4/18/23
to Prometheus Users
The Pushgateway's documentation includes the ability to delete all metrics for a given job and instance label.


The Prometheus go client has support for making a pusher for a specific job label, but doesn't appear to have anywhere to specify the instance label. Does anyone know if there is a way, using the go library, to delete metrics for a specific job and instance label pair?

Thanks,
Aaron

Julius Volz

unread,
Apr 19, 2023, 8:46:00 AM4/19/23
to Aaron Reisman, Prometheus Users
Yes, you can use the Grouping() method to add other labels to the grouping key: https://pkg.go.dev/github.com/prometheus/client_golang/prometheus/push#Pusher.Grouping - it returns a new Pusher with the new label added.

Note that instance-level usage of the Pushgateway is not usually encouraged (see https://prometheus.io/docs/practices/pushing/#when-to-use-the-pushgateway for why).

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6e8eadf6-cba1-4838-afce-1f0546805674n%40googlegroups.com.


--
Julius Volz
PromLabs - promlabs.com

Aaron Reisman

unread,
Apr 20, 2023, 5:09:05 PM4/20/23
to Prometheus Users
The Grouping() method doesn't seem to leverage the instance id in the url of the push gateway. In my use case I've inherited a batch framework where the batch jobs push metrics to a common pushgateway solution, but nothing is deleting them. I'm trying to write a service that will track the batch jobs and delete the metrics after a period of time where the batch job type was used as the job id and the specific instance of the batch job is used as the instance id.

Having read through a fair amount of documentation, this setup seems like an appropriate use of instance level metrics. I'd be happy to hear if there is a better way to handle it. The best practices linked in the response seems to indicate that maybe it shouldn't be used here, but it doesn't cover what would be recommended instead.

Julius Volz

unread,
Apr 21, 2023, 4:29:46 PM4/21/23
to Aaron Reisman, Prometheus Users
On Thu, Apr 20, 2023 at 11:09 PM Aaron Reisman <pix...@gmail.com> wrote:
The Grouping() method doesn't seem to leverage the instance id in the url of the push gateway.

So this works for me and correctly groups the metrics under both the job and instance label:


Having read through a fair amount of documentation, this setup seems like an appropriate use of instance level metrics. I'd be happy to hear if there is a better way to handle it. The best practices linked in the response seems to indicate that maybe it shouldn't be used here, but it doesn't cover what would be recommended instead.

Hmm, yeah it's ok-ish, you'll just have to automate the instance-level metrics deletion yourself (as you are trying to do).
 
On Wednesday, April 19, 2023 at 7:46:00 AM UTC-5 Julius Volz wrote:
Yes, you can use the Grouping() method to add other labels to the grouping key: https://pkg.go.dev/github.com/prometheus/client_golang/prometheus/push#Pusher.Grouping - it returns a new Pusher with the new label added.

Note that instance-level usage of the Pushgateway is not usually encouraged (see https://prometheus.io/docs/practices/pushing/#when-to-use-the-pushgateway for why).

On Tue, Apr 18, 2023 at 3:36 PM Aaron Reisman <pix...@gmail.com> wrote:
The Pushgateway's documentation includes the ability to delete all metrics for a given job and instance label.


The Prometheus go client has support for making a pusher for a specific job label, but doesn't appear to have anywhere to specify the instance label. Does anyone know if there is a way, using the go library, to delete metrics for a specific job and instance label pair?

Thanks,
Aaron

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6e8eadf6-cba1-4838-afce-1f0546805674n%40googlegroups.com.


--
Julius Volz
PromLabs - promlabs.com

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages