Pods stuck in "Unknown" status

[BG]

I see this happen a lot and I've never been able to find a solution.

A pod (from a Kubernetes Job Deployment) will be an "Unknown" status. If I run `kubectl delete pod <podname>` I get "<podname> deleted" but the pod is still there, in an unknown state.

Kubernetes Version: 1.7.3

kubectl version:
client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.3", GitCommit:"029c3a408176b55c30846f0faedf56aae5992e9b", GitTreeState:"clean", BuildDate:"2017-02-22T10:12:27Z", GoVersion:"go1.8", Compiler:"gc", Platform:"darwin/amd64"}

[Rodrigo Campos]

It's quite broad and vague what we can see from your email, so it's really difficult to help. Can you please elaborate and give as much detail as possible and what things you tried and didn't work, etc.?

For example, What about Kubectl describe? And Log? Does it have PVs or something else? Can you create pods on some other way? Etc, etc.

--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[griff...@gmail.com]

There isn't a lot of information to work with here, Roridgo. It's a vague description because Kubernetes is being vague about this error.

Calling `kubectl log` shows the logs from the container IF I have no not deleted the pod. If I delete the pod, calling `kubectl logs` on the pod that comes back returns blank.

Calling `kubectl describe will sometimes return the usual description of the pod, other times it returns nothing at all. I haven't noticed what sequence of steps causes one to happen over the other.

[griff...@gmail.com]

Also, I don't know what you mean by "can you create pods by some other way" - I create the pods the exact same way each time. `kubectl create -f file.yml`, albiet done programmatically (a process dynamically writes the file and calls kubectl create).

I run several thousand short-lived processes via Jobs a day and very rarely hit this. But when I do, the pods linger around for days in an Unknown status. It seems like this is very clearly a bug unless there is better documentation/explanation around what Unknown status means.

[Rodrigo Campos]

Well, that is considerable way more info. If you can find a way to reproduce, then you will have a perfect bug report to open an issue on GitHub. If not, probably open but try to find one. You are convinced it's a bug, after all :-)

Also, have you tried looking at logs or using more verbose logging when this happens? More info to get a clearer picture is always good :)