Error: dial tcp getsockopt: connection timed out

552 views
Skip to first unread message

Rares Vernica

unread,
May 21, 2018, 3:50:31 PM5/21/18
to Kubernetes user discussion and Q&A
Hello,

I have a nine bare metal hosts setup with CentOS Atomic. I followed their guide on how to setup Kubernetes from
http://www.projectatomic.io/docs/gettingstarted/ I successfully created a test service running Nginx and I was able to access it.

Moving further, I got the kubernetes-dashboard running but I get some network errors when I assess it. I wonder if missed some of the network settings. Here is what I did:

> kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.5.1/src/deploy/kubernetes-dashboard.yaml

> kubectl -n kube-system get service kubernetes-dashboard
NAME                   CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
kubernetes-dashboard   10.0.0.184   <nodes>       80:31247/TCP   57m

> kubectl -n kube-system log kubernetes-dashboard-3203831700-lqmnb
Using HTTP port: 9090
Creating API server client for https://10.0.0.1:443
Successful initial request to the apiserver, version: v1.5.2
Creating in-cluster Heapster client
Using service account token for csrf signing

> kubectl -n kube-system describe services kubernetes-dashboard
Name:            kubernetes-dashboard
Namespace:        kube-system
Labels:            app=kubernetes-dashboard
Selector:        app=kubernetes-dashboard
Type:            NodePort
IP:            10.0.0.184
Port:            <unset>    80/TCP
NodePort:        <unset>    31247/TCP
Endpoints:        172.16.74.2:9090
Session Affinity:    None
No events.

> kubectl proxy
Starting to serve on 127.0.0.1:8001

When I try to access it I get:

> curl http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard/
Error: 'dial tcp 172.16.74.2:9090: getsockopt: connection timed out'
Trying to reach: 'http://172.16.74.2:9090/'

> ping 172.16.74.1
PING 172.16.74.1 (172.16.74.1) 56(84) bytes of data.
64 bytes from 172.16.74.1: icmp_seq=1 ttl=64 time=0.415 ms

> ping 172.16.74.2
PING 172.16.74.2 (172.16.74.2) 56(84) bytes of data.
From 172.16.5.0 icmp_seq=10 Destination Host Unreachable

I can see the dashboard container is running, but on one of the eight Kubernetes nodes (not the master):

node> docker ps
CONTAINER ID        IMAGE                                                        COMMAND                  CREATED             STATUS              PORTS               NAMES
6398a6f87004        gcr.io/google_containers/kubernetes-dashboard-amd64:v1.5.1   "/dashboard --port..."   About an hour ago   Up About an hour                        k8s_kubernetes-dashboard.d748d820_kubernetes-dashboard-3203831700-lqmnb_kube-system_e26bfb3b-5ad4-11e8-9cee-001f29049d2f_528eaea4
ae8109a36917        registry.access.redhat.com/rhel7/pod-infrastructure:latest   "/usr/bin/pod"           About an hour ago   Up About an hour                        k8s_POD.28c50bab_kubernetes-dashboard-3203831700-lqmnb_kube-system_e26bfb3b-5ad4-11e8-9cee-001f29049d2f_6ddfaedd

node> docker exec -ti ae8109a36917 cat /etc/hosts
...
172.16.74.2     kubernetes-dashboard-3203831700-lqmnb

node> ping 172.16.74.2
PING 172.16.74.2 (172.16.74.2) 56(84) bytes of data.
64 bytes from 172.16.74.2: icmp_seq=1 ttl=64 time=0.053 ms

node> curl http://172.16.74.2:9090
<!doctype html> <html ng-app="kubernetesDashboard">...

It's interesting how the master can't connect to the IP of the container. I wonder what I'm missing here.

Thanks!
Rares

Rares Vernica

unread,
May 24, 2018, 4:59:04 PM5/24/18
to Kubernetes user discussion and Q&A
I figured out the problem, flanneld was not running on the master node.

Cheers,
Rares
Reply all
Reply to author
Forward
0 new messages