My prometheus is suddenly seeing 2 pods as up == 0 - but when I look at them - they have status Running 3/3 and Running 2/2 respectively - so they seem just fine.
How can I debug how they end up with having the up metric set to 0?
promql query: up == 0
returns:
up{app="dynamic-gateway-service",controller_revision_hash="rcd4359800f-dynamic-gateway-service-898c67dc8",instance="
192.168.26.20:63512",job="kubernetes-pods",kubernetes_namespace="api-klavs-gwb-d1",kubernetes_pod_name="rcd4359800f-dynamic-gateway-service-0",release="rcd4359800f",statefulset_kubernetes_io_pod_name="rcd4359800f-dynamic-gateway-service-0"}
up{app="dynamic-gateway-service",controller_revision_hash="r7fdf61c6a9-dynamic-gateway-service-8867b4f6c",instance="
192.168.97.149:63512",job="kubernetes-pods",kubernetes_namespace="api-klavs-gwa-d1",kubernetes_pod_name="r7fdf61c6a9-dynamic-gateway-service-0",release="r7fdf61c6a9",statefulset_kubernetes_io_pod_name="r7fdf61c6a9-dynamic-gateway-service-0"}
and kubectl get pods in klavs-api-gwa has this:
r7fdf61c6a9-dynamic-gateway-service-0 3/3 Running 0 21h