How to collect metrics from multiple containers in a pod that have a /metrics endpoint

[karthik...@gmail.com]

Hello,

I have a pod with 3 containers that are all currently have a metrics endpoint that send application related data. I know I can use the prometheus pusher->push gateway to scrape the values of these nodes and send them over.

I'm wondering how to do this in the pull model with prometheus trying to scrap the values. Can I use the prometheus blackbox-exporter for this purpose to scrape the /metrics endpoint of all the containers and send them over as a batch?

Thanks!

[karthik...@gmail.com]

Forgot to specify that I'm my application in kubernetes- so it's 1 deployment with 2 pods that have multiple containers mapped to each of the pods. All the containers in the pod have a /metrics endpoint mapped to them.

[clap...@gmail.com]

Prometheus is based on the pull model - ideally you should get Prometheus to pull directly from each of your containers. The push gateway should be used for things like short lived jobs, and the blackbox exporter is for a completely different use case.

Have you checked out the documentation for service discovery using kubernetes? https://prometheus.io/docs/prometheus/latest/configuration/configuration/#<kubernetes_sd_config> 

[Karthik Gaekwad]

Thanks! That is super useful, and will have to read more about it. It looks like what I'm looking for is the the pod config (https://prometheus.io/docs/prometheus/latest/configuration/configuration/#pod). It sounds like I can have multiple of these for a pod per "For each declared port of a container, a single target is generated"

• __meta_kubernetes_pod_container_port_number: Number of the container port 
  

Does that mean I'd just declare whatever container port numbers are available to receive data, and prometheus will do the work to scrape each endpoint in the pod?

Thanks!

-Karthik

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/nRnqWbIrGMI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/159fd2f8-76a3-4481-aa7a-54ac5ce53699%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

-Karthik Gaekwad
http://about.me/kgaekwad