kube-dns goes into CrashLoopBackOff state after starting flannel
892 views
Skip to first unread message
Ajit Sarnaik
Nov 20, 2017, 6:36:47 AM11/20/17
to kubernetes-sig-network
Have started creating a kubernetes cluster, what I am noticing is that when I create the cluster using "kubeadmn init" with the podcidr, the pods start up and kube-dns goes into pending state. Next when I start flannel and rbac, the kube-dns goes to running state. But a short time later, it goes into "CrashLoopBackOff" state. Has anybody seen this behavior and any solutions please. Appreciate the help.
The events that I notice are as follows,
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 19m (x72 over 40m) default-scheduler No nodes are available that match all of the predicates: NodeNotReady (1).
Normal SuccessfulMountVolume 15m kubelet, ip-172-31-51-135 MountVolume.SetUp succeeded for volume "kube-dns-config"
Normal SuccessfulMountVolume 15m kubelet, ip-172-31-51-135 MountVolume.SetUp succeeded for volume "kube-dns-token-xrh7v"
Normal Pulling 15m kubelet, ip-172-31-51-135 pulling image "gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.5"
Normal Pulled 15m kubelet, ip-172-31-51-135 Successfully pulled image "gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.5"
Normal Pulling 15m kubelet, ip-172-31-51-135 pulling image "gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.5"
Normal Pulling 15m kubelet, ip-172-31-51-135 pulling image "gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.5"
Normal Started 15m kubelet, ip-172-31-51-135 Started container
Normal Pulled 15m kubelet, ip-172-31-51-135 Successfully pulled image "gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.5"
Normal Created 15m kubelet, ip-172-31-51-135 Created container
Normal Pulled 15m kubelet, ip-172-31-51-135 Successfully pulled image "gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.5"
Normal Created 15m kubelet, ip-172-31-51-135 Created container
Normal Started 15m kubelet, ip-172-31-51-135 Started container
Normal Started 14m (x2 over 15m) kubelet, ip-172-31-51-135 Started container
Normal Created 14m (x2 over 15m) kubelet, ip-172-31-51-135 Created container
Normal Pulled 14m kubelet, ip-172-31-51-135 Container image "gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.5" already present on machine
Warning Unhealthy 14m (x7 over 15m) kubelet, ip-172-31-51-135 Readiness probe failed: Get http://10.244.0.2:8081/readiness: dial tcp 10.244.0.2:8081: getsockopt: connection refused
Warning Unhealthy 5m (x18 over 14m) kubelet, ip-172-31-51-135 Liveness probe failed: HTTP probe failed with statuscode: 503
Warning FailedSync 23s (x37 over 12m) kubelet, ip-172-31-51-135 Error syncing pod
A snipet of the log is shown below,
I1120 11:31:20.209231 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:20.709233 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:21.209234 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:21.709299 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:22.209233 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:22.709268 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:23.209242 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:23.709233 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:24.209231 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:24.709236 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:25.209334 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:25.709182 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:26.209302 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
I1120 11:31:26.709244 1 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
F1120 11:31:27.209237 1 dns.go:168] Timeout waiting for initialization
######################################################################################
Below are the versions that I am using,
################################################
Docker:
Client:
Version: 17.10.0-ce
API version: 1.33
Go version: go1.8.3
Git commit: f4ffd25
Built: Tue Oct 17 19:04:16 2017
OS/Arch: linux/amd64
Server:
Version: 17.10.0-ce
API version: 1.33 (minimum version 1.12)
Go version: go1.8.3
Git commit: f4ffd25
Built: Tue Oct 17 19:02:56 2017
OS/Arch: linux/amd64
Experimental: false
##################################################
Kubernetes:
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-08T18:39:33Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-08T18:27:48Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
##################################################
Flannel:
Ajit Sarnaik
Nov 20, 2017, 10:39:24 AM11/20/17
to kubernetes-sig-network
Prateek Gogia
Nov 20, 2017, 1:03:44 PM11/20/17
to Ajit Sarnaik, kubernetes-sig-network
I see this in your output -
Also, your Docker version is 17.X. You may want to double check your iptables rule for FORWARD chain. If it is DROP. You can fix it by running - iptables -P FORWARD ACCEPT
--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-network+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-network@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-sig-network.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages