kube-state-metrics: count number of pods grouped by phase
3,315 views
Skip to first unread message
Poussier William
Jun 2, 2020, 12:48:03 PM6/2/20
to Prometheus Users
Hello,
I am trying to represent in a Grafana dahboard the cumulative count of all Kubernetes pod phases per a list of nodes.
The metrics uses are from the kube-state-metrics project: https://github.com/kubernetes/kube-state-metrics/blob/v1.9.4/docs/pod-metrics.md
- kube_pod_info
- kube_pod_status_phase
The kube_pod_status_phase serie does not have a node label, while the kube_pod_info does.
I have tried to join the two series using the group_left function of Prometheus, but it seems that I can't conditionally select on the joined label.
My query currently looks like the following:
sum(sum(kube_pod_info{cluster_id="$region",node=~"$node"} * on(pod, namespace) group_right(node) kube_pod_status_phase{cluster_id="$region"}) by (node,phase)) by (phase)
I get the following as a result (taken from the Grafana panel legend):
I would like to count the number of pods (according to the selected nodes) grouped by pod phase.
What do I need to change to achieve that ?
Thanks
William
Poussier William
Jun 2, 2020, 6:35:08 PM6/2/20
to Prometheus Users
I have simplified the query to the following:
sum(kube_pod_status_phase{cluster_id="$region"} * on (namespace,pod) group_left(node) kube_pod_info{cluster_id="$region",node=~"$node"}) by (node,phase)Still facing the same issue, there is two series in the return.
Grafana is displaying the following as table:
The values of the kube_pod_info series here are actually representing the right data value per node for the "Running" phase.
Matthias Rampke
Jun 3, 2020, 3:02:57 AM6/3/20
to Poussier William, Prometheus Users
The second query looks right (and works if I plug it into my Prometheus) – I don't know what Grafana is doing with it though. I would recommend solving one problem at a time; does the query work as you expect when you use the built in Prometheus query UI? If so, it is a problem with the panel configuration. If not, you can tweak it there, then come back to the panel once you have the underlying data in shape.
/MR
--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/aede4847-2944-42b1-bbb9-fa1aeffdeed2%40googlegroups.com.
Poussier William
Jun 3, 2020, 7:17:27 AM6/3/20
to Prometheus Users
I don't have access to a Prometheus UI, the underlying platform uses Warp10, and my queries are transformed from PromQL to WarpScript.
I successfully obtained the desired output using the following query:
max(count(kube_pod_info{cluster_id="$region",node=~"$node"}) * on(pod,namespace) group_left(phase) kube_pod_status_phase{cluster_id="$region"}) by (phase)Le mercredi 3 juin 2020 09:02:57 UTC+2, Matthias Rampke a écrit :
The second query looks right (and works if I plug it into my Prometheus) – I don't know what Grafana is doing with it though. I would recommend solving one problem at a time; does the query work as you expect when you use the built in Prometheus query UI? If so, it is a problem with the panel configuration. If not, you can tweak it there, then come back to the panel once you have the underlying data in shape./MR
On Tue, Jun 2, 2020 at 10:35 PM Poussier William <william...@gmail.com> wrote:
--I have simplified the query to the following:
Still facing the same issue, there is two series in the return.Grafana is displaying the following as table:
The values of the kube_pod_info series here are actually representing the right data value per node for the "Running" phase.
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages