OpenResty in Kubernetes, DNS and resolv.conf issues

[gmc...@zendesk.com]

This post refers to latest OpenResty image available at:

https://github.com/openresty/docker-openresty/blob/master/stretch/Dockerfile.fat

I'm trying to use OpenResty in a Kubernetes cluster. The code relies on coroutines, which call out to proxy_pass locations.

The hosts in proxy_pass directives are Kubernetes service names, but the namespace differs depending on the environment. The code therefore needs to parse the search list in resolv.conf in order to fully expand the hostname for the relevant namespace.

eg

proxy_pass http://my-service/api;

# cat /etc/resolv.conf

nameserver 169.254.1.1

search staging.svc.cluster.local svc.cluster.local cluster.local

So, request to 

my-service

should be expanded to 

my-service.staging.svc.cluster.local

I have verified from the OS layer than the search list is working as expected.

# ping my-service

PING my-service (10.231.64.104): 56 data bytes

I have tried to configure OpenResty/Nginx as follows to make it parse /etc/resolv.conf, as per docs (https://github.com/openresty/openresty/#resolvconf-parsing):

resolver 169.254.1.1 local=on local=/etc/resolv.conf ipv6=off;

However, in my Lua logs, I see the following error:

[error] 8#8: *1 my-service could not be resolved (3: Host not found)

[Thibault Charbonnier]

Hello,

As far as I recall, the NGINX resolver does not support search domains,
and thus, the resolver_conf_parsing patch[1] doesn't either.

I am not sure if NGINX Plus supports it. Possibly, considering the
resolver bundled with it supports SRV records[2], but I don't have
access to an NGINX Plus instance to try it.

My suggestion would be to run dnsmasq[3] alongside OpenResty and
configure the NGINX 'resolver' directive to point to it, like so:

resolver 127.0.0.1:53

Or using a pure Lua-land DNS resolver, such as:

https://github.com/Kong/lua-resty-dns-client

It supports search domains and various resolv.conf options (see 'man 5
resolv.conf'), even if specified as environment variables.

Best,
Thibault

[1]:
https://github.com/openresty/openresty/blob/master/patches/nginx-1.15.8-resolver_conf_parsing.patch
[2]: https://www.nginx.com/blog/dns-service-discovery-nginx-plus/
[3]: http://thekelleys.org.uk/dnsmasq/doc.html

> --
> You received this message because you are subscribed to the Google
> Groups "openresty-en" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openresty-en...@googlegroups.com
> <mailto:openresty-en...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[边冀]

When local=on, the standard path of /etc/resolv.conf will be used. You may also specify arbitrary path to be used for parsing, for example: local=/tmp/test.conf

在 2019年5月3日星期五 UTC+8下午10:40:06，gmc...@zendesk.com写道：