Kubernetes API exporter

[Tom Wilkie]

Hello!

I've built a small Prometheus exporter for gauges in the Kubernetes API:

https://github.com/tomwilkie/kube-api-exporter

I was trying to solve the problem of getting alerts for when Deployments fail.  I couldn't find the metrics exposed natively by Kubernetes itself, but I would be interested to hear how other people have solved this problem.

Any other feedback would be very welcome!

Thanks

Tom

[Tobias Schmidt]

Thanks! We've a similar needs and were talking about a kubernetes
exporter as well. We wish to have metrics on pod restarts for example.

So far we don't collect any such metrics with Prometheus. One idea was
to add a second monitoring endpoint to the kubelet which could export
such meta information.

> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-devel...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Tom Wilkie]

Pod restarts is a fantastic example. We're going to add something for crash looping pods to. 

I'll put together something for pods tomorrow. 

Thanks

Tom

[frederic...@coreos.com]

Hi there,

we're actually looking into adding some additional metrics into kubernetes itself. These are great examples! We'll try to get some of these into kubernetes directly. Thanks for collecting these metrics that are relevant in real scenarios, it really gives us a good idea of what to implement and a head start in terms of investigation ;)

Cheers
Frederic

[Fabian Reinartz]

For context: Frederic is our latest addition to the monitoring team at CoreOS :)

[Tom Wilkie]

> we're actually looking into adding some additional metrics into kubernetes itself. These are great examples! We'll try to get some of these into kubernetes directly. Thanks for collecting these metrics that are relevant in real scenarios, it really gives us a good idea of what to implement and a head start in terms of investigation ;)

That would be awesome!  Let me know if I can help.

Thanks

Tom

[Romain Vrignaud]

Hi guys,

We are using kube-state-metrics : https://github.com/kubernetes/kube-state-metrics for this (pod restart)

[Tom Wilkie]

Excellent - I wish I'd know about that before I coded up kube-api-exporter.  Will give it a try.

Thanks

Tom

[Bryce Walter]

It would be great if Prometheus could be set up to use kubectl or kubelet with their kube-configs. This would make gathering metrics a lot easier. Especially now that Kubernetes has branched off SkyDNS/Kube2Sky in favor of kube-dns. During the transition, they seemed to have dropped the code that probes and scrapes much needed ClusterDNS
Data. I know @thawkin posted about it last week, but don't see any movement on it yet.
The new service doesn't have the /metrics endpoint.