hostname not resolved inside a container

2,395 views
Skip to first unread message

dan.ste...@googlemail.com

unread,
Jun 25, 2018, 11:35:43 PM6/25/18
to Kubernetes user discussion and Q&A
Hello,

I am not quite sure if this a problem with the configuration how we deploy a new cluster or we just not finish the configuration after the cluster is deployed. But at the moment the hostname inside the pods are not resolving while the using of IPs works fine, both (cluster and pods) get routable IPs in our company network that's why I am a little bit confused why the request not reach the pods. Did I need so setup something like kubedns? So the request first ask the kubedns (in the cluster) and this one gives the request to our company dns-server?
best regards
Dan

dan.ste...@googlemail.com

unread,
Jun 26, 2018, 1:44:11 AM6/26/18
to Kubernetes user discussion and Q&A
ok now I take a longer look around and it seems kubespray (the way we deploy our new cluster) already brings kube-dns to the cluster. In a pod I do for another pod:

nslookup mysql
Server: 10.233.0.3
Address 1: 10.233.0.3 kube-dns.kube-system.svc.example.com

Name: mysql
Address 1: 10.233.52.31 mysql.default.svc.example.com


but if I do for an server in our compaony network. I get this

# nslookup ldap
Server: 10.233.0.3
Address 1: 10.233.0.3 kube-dns.kube-system.svc.example.com
nslookup: can't resolve 'ldap'

I already at our upstream-Nameservers by:

apiVersion: v1
kind: ConfigMap
metadata:
name: kube-dns
namespace: kube-system
data:
upstreamNameservers: |
["8.8.8.8"]

but this doesn't change the result :-(

Message has been deleted

dan.ste...@googlemail.com

unread,
Jun 26, 2018, 5:15:13 AM6/26/18
to Kubernetes user discussion and Q&A
Ok I think I take another step if I do nslookup for a server which has the them domain like the cluster it couldn't resolved but if I make a lookup for an server which belongs to another subdomain like foo.example.com it works! :-)

The question I am facing now is why gives kube-dns the request for servernamens he doesn't know not to the upstream-nameserver?

Tim Hockin

unread,
Jun 26, 2018, 12:59:33 PM6/26/18
to Kubernetes user discussion and Q&A
On Mon, Jun 25, 2018 at 10:44 PM dan.steffen.de via Kubernetes user discussion and Q&A <kubernet...@googlegroups.com> wrote:
ok now I take a longer look around and it seems kubespray (the way we deploy our new cluster) already brings kube-dns to the cluster. In a pod I do for another pod:

nslookup mysql
Server:    10.233.0.3
Address 1: 10.233.0.3 kube-dns.kube-system.svc.example.com

Name:      mysql
Address 1: 10.233.52.31 mysql.default.svc.example.com

To be pedantic, you looked up a Service, not a Pod.   And while you specified "mysql", the "search" directove of resolv.conf expanded that to "mysql.default.svc.example.com", which kube-dns know how to answer.

I say this to give context to the failure (I hope)
 
but if I do for an server in our compaony network. I get this

# nslookup ldap
Server:    10.233.0.3
Address 1: 10.233.0.3 kube-dns.kube-system.svc.example.com
nslookup: can't resolve 'ldap'

is "ldap" a fully qualified name (hint: it can't be - there are no dots).  Since it is not FQDN, it also uses "search".  resolv.conf in a pod is preconfigured to search kubernetes names and then to inherit the search from your VM.  E.g. on a GCE VM, my search paths list is "default.svc.cluster.local   svc.cluster.local   cluster.local   c.thockin-dev.internal   google.internal".  If you look at yours, does any of the "search" line cover the domain you need to make "ldap" into an FQDN?
 
I already at our upstream-Nameservers by:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-dns
  namespace: kube-system
data:
  upstreamNameservers: |
    ["8.8.8.8"]

8.8.8.8 is public - it won't know anything about your internal names.
 

dan.ste...@googlemail.com

unread,
Jun 27, 2018, 2:47:40 AM6/27/18
to Kubernetes user discussion and Q&A
First of all I would like to thank you for your response and the hint that I don't try to resolve the name by the pod rather for the service.

Maybe my post is at this point uncomplete I just post the result of the nslookup for ldap because the domain example.com exists in the search-tag of the /etc/resolv.conf on the node and in the pod too. But to complete this point the nslookup don't work with just the hostname nor the FQDN.

The strange thing I am facing at the moment is that a nslookup for webserver.foobar.example.com works that's why it look for me at the moment that the pods ask the kube-dns: "do you know ldap.example.com?" (the example.com is the domain in which the cluster lives too) and if kube-dns didn't know this host the nameserver don't give the request to the upstream nameserver and says: "can't resolve". But when a server has a request for a subdomain which isn't known by the DNS the nameserver of kubernetes gives this request to the upstream-nameserver.
If this is really the problem, how can I configure kube-dns that he ask the upstream nameserver if he doesn't know the host.

The explanation for your second hint is, that I don't want to post here the (private) IPs of our company network that's why I choose this one (and copied it from a HowTo) but maybe I should for this problem/example the IP 192.168.1.5 as the nameserver of your company

Tim Hockin

unread,
Jun 28, 2018, 2:56:11 PM6/28/18
to Kubernetes user discussion and Q&A
On Tue, Jun 26, 2018 at 11:47 PM dan.steffen.de via Kubernetes user discussion and Q&A <kubernet...@googlegroups.com> wrote:
First of all I would like to thank you for your response and the hint that I don't try to resolve the name by the pod rather for the service.

Maybe my post is at this point uncomplete I just post the result of the nslookup for ldap because the domain example.com exists in the search-tag of the /etc/resolv.conf on the node and in the pod too. But to complete this point the nslookup don't work with just the hostname nor the FQDN.

The strange thing I am facing at the moment is that a nslookup for webserver.foobar.example.com works that's why it look for me at the moment that the pods ask the kube-dns: "do you know ldap.example.com?" (the example.com is the domain in which the cluster lives too) and if kube-dns didn't know this host the nameserver don't give the request to the upstream nameserver and says: "can't resolve". But when a server has a request for a subdomain which isn't known by the DNS the nameserver of kubernetes gives this request to the upstream-nameserver.
If this is really the problem, how can I configure kube-dns that he ask the upstream nameserver if he doesn't know the host.

 
If you configure a cluster to serve on "example.com" it assumes it owns ALL of example.com.  You can't have other names in example.com because they potentially collide.  The DNS server becomes "authoritative" for the domain you give it.  In this case, if you ask for "ldap.example.com" and kube-dns does not know it, it assumes that name does not exist and returns NXDOMAIN.

If you need to interop with an existing domain, you can give your cluster a subdomain.  e.g. k8s.example.com for your cluster and example.com for your company.
 
The explanation for your second hint is, that I don't want to post here the (private) IPs of our company network that's why I choose this one (and copied it from a HowTo) but maybe I should for this problem/example the IP 192.168.1.5 as the nameserver of your company

We all use the same private IPs.  But I respect your desire to not expose that.  I just misunderstood :)
 



--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages