kubectl asks "Please enter Username:" over and over and gives bad results

[Ravi Kapoor]

I am facing this really weird behavior. Kubectl suddently started asking for uid/pwd.

Even when I enter it, it asks for it several times (see first command)

and sometimes gives results that make no sense e.g. claims there are no pods running (see second command)

what am I doing wrong?

root@ip-x-x-x-x:~# kubectl cluster-info

Please enter Username: admin

Please enter Password: ********************************

Kubernetes master is running at https://api.k8s.example.com

Please enter Username: admin

Please enter Password: ********************************

Please enter Username: admin

Please enter Password: ********************************

Elasticsearch is running at https://api.k8s.example.com/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging

Heapster is running at https://api.k8s.example.com/api/v1/proxy/namespaces/kube-system/services/heapster

Kibana is running at https://api.k8s.example.com/api/v1/proxy/namespaces/kube-system/services/kibana-logging

KubeDNS is running at https://api.k8s.example.com/api/v1/proxy/namespaces/kube-system/services/kube-dns

Grafana is running at https://api.k8s.example.com/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana

InfluxDB is running at https://api.k8s.example.com/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

root@ip-x-x-x-x:~# kubectl cluster-info

Please enter Username: ^C

root@ip-x-x-x-x:~# kubectl get pods

Please enter Username: admin

Please enter Password: ********************************

Please enter Username: admin

Please enter Password: ********************************

No resources found.

[Eric Tune]

 If you use GKE try repeating the gcloud command to get your credentials.  If you load creds another way, try repeating that.  If neither,  check your .kube/config file and make sure the current-context: points to a valid context record, and that context record points to a valid user record with your name a password, or whatever you use to auth. 

--
You received this message because you are subscribed to the Google Groups "Kubernetes developer/contributor discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-dev+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-dev@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAF65o-tHkK87UeJgZBdkTHPU3t1gYg9xyoPvixDWD_rA4iPj1A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Ravi Kapoor]

I am using AWS. My credentials are correct. If you see my command output, after entering password 3 times, it does show (almost) correct output of kubectl cluster-info.

Somehow it does not show dashboard in it. I tried to install dashboard again and got error "already exists".

So I tried to go to dashboard URL https://api.k8s.example.com/ui and get following error on browser

   Error: 'dial tcp 10.123.45.2:9090: getsockopt: connection refused'

   Trying to reach: 'http://10.123.45.2:9090/'

The dashboard has vanished on me before and somehow magically came back after I resized my cluster.

To post to this group, send email to kuberne...@googlegroups.com.

[Ravi Kapoor]

Update: 

since I am using kops, I edited node sizes to force moving of pods.

after I did rolling update and new nodes were up, my kubectl command started working.

I had to delete and reinstall dashboard to make it work.

Overall very worrisome behavior as I cannot change node sizes in prod

[Mike Danese]

Please file an issue against kops. 

To post to this group, send email to kubernetes-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAF65o-tS4hgdx226-h75TTZUKnPK5JN5oVDVAmd8-N3EuxJkLQ%40mail.gmail.com.

[Ravi Kapoor]

Mike the primary issue is that it is not reproducible. My cluster has been sitting idle for 3 weeks and I faced these issues. Should I still file it?

I can report again if/when it happens again and I can work with someone to debug it to figure out what is going on. 

[Mike Danese]

I'm sure kops devs would like to hear regardless. 

To post to this group, send email to kubernetes-dev@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAF65o-vh_%2Bm4%2B3XYs3rkjVAJVi1Vcg_1s81LSb0_TN5S1yB9zw%40mail.gmail.com.

[Ravi Kapoor]

Done. https://github.com/kubernetes/kops/issues/1449

thanks