DNS and domains "openvpn" and "local"

[Nicholas]

I have a router/firewall running an OpenVPN server through the Vyatta
Core router software. I'm running the Tunnelblick client on Mac OS X
Snow Leopard. Almost everything is working perfectly, but I'm having
some problems with the search domain in DNS. My Vyatta configuration
looks like this (I know it's not a straight OpenVPN configuration, but
you should be able to gleam most things off of it):

vyatta@fw# show interfaces openvpn
openvpn vtun0 {
mode server
server {
name-server 172.16.1.1
push-route 192.168.1.254/24
push-route 192.168.122.254/24
subnet 172.16.1.0/24
topology subnet
}
tls {
ca-cert-file /root/openvpn/keys/ca.crt
cert-file /root/openvpn/keys/server.crt
dh-file /root/openvpn/keys/dh1024.pem
key-file /root/openvpn/keys/server.key
}
}
[edit]

It pushes everything properly. The routes work. I can access all the
servers behind the VPN that I'm supposed to be able to using their IP
addresses from any software. The DNS server is even correctly set:

Legolas:~ Tim$ cat /etc/resolv.conf
domain openvpn
search openvpn
nameserver 172.16.1.1
Legolas:~ Tim$ scutil --dns
DNS configuration
resolver #1
domain : openvpn
search domain[0] : openvpn
nameserver[0] : 172.16.1.1
order : 200000
...

However, the default "openvpn" search domain made it impossible for me
to resolve hostname-only machines behind the firewall ("lynyrd",
"garth", etc.). I couldn't figure out how to make it not set ANY
search domain, so I pushed the local domain:

set interfaces openvpn vtun0 openvpn-option "--push dhcp-option DOMAIN
local"

And it gets pushed successfully:

Legolas:~ Tim$ more /etc/resolv.conf
domain local
search local
nameserver 172.16.1.1
Legolas:~ Tim$ scutil --dns
DNS configuration
resolver #1
domain : local
search domain[0] : local
nameserver[0] : 172.16.1.1
order : 200000

After adding ".local" versions of my hostname DNS entries, I can
successfully lookup records for "lynyrd(.local)," etc. However, I
still cannot access the machine using the hostname from any software
(only the IP):

Legolas:~ Tim$ ping lynyrd
ping: cannot resolve lynyrd: Unknown host
Legolas:~ Tim$ host lynyrd
lynyrd.local has address 192.168.122.101
Legolas:~ Tim$ ping 192.168.122.101
PING 192.168.122.101 (192.168.122.101): 56 data bytes
64 bytes from 192.168.122.101: icmp_seq=0 ttl=63 time=37.013 ms
64 bytes from 192.168.122.101: icmp_seq=1 ttl=63 time=31.014 ms
64 bytes from 192.168.122.101: icmp_seq=2 ttl=63 time=36.146 ms
^C
--- 192.168.122.101 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 31.014/34.724/37.013/2.647 ms

Legolas:~ Tim$ ping lynyrd.local
ping: cannot resolve lynyrd.local: Unknown host
Legolas:~ Tim$ host lynyrd.local
lynyrd.local has address 192.168.122.101
Legolas:~ Tim$ ping 192.168.122.101
PING 192.168.122.101 (192.168.122.101): 56 data bytes
64 bytes from 192.168.122.101: icmp_seq=0 ttl=63 time=37.013 ms
64 bytes from 192.168.122.101: icmp_seq=1 ttl=63 time=31.014 ms
64 bytes from 192.168.122.101: icmp_seq=2 ttl=63 time=36.146 ms
^C
--- 192.168.122.101 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 31.014/34.724/37.013/2.647 ms

If I try to go to 192.168.122.101 in Safari, I get the web page I'm
supposed to. If I try to go to "lynyrd" or "lynyrd.local," it can't
find the server.

I've worked on this for a few hours now, but I'm at a loss. Banging my
head against the wall isn't going to help any more. Any suggestions/
answers?

Thanks in advance,

Nick

Full scutil output:

Legolas:~ Tim$ scutil --dns
DNS configuration

resolver #1
domain : local
search domain[0] : local
nameserver[0] : 172.16.1.1
order : 200000

resolver #2
domain : local
options : mdns
timeout : 2
order : 300000

resolver #3
domain : 254.169.in-addr.arpa
options : mdns
timeout : 2
order : 300200

resolver #4
domain : 8.e.f.ip6.arpa
options : mdns
timeout : 2
order : 300400

resolver #5
domain : 9.e.f.ip6.arpa
options : mdns
timeout : 2
order : 300600

resolver #6
domain : a.e.f.ip6.arpa
options : mdns
timeout : 2
order : 300800

resolver #7
domain : b.e.f.ip6.arpa
options : mdns
timeout : 2
order : 301000

[jkbull...gmail.com]

A couple of things:

Make sure you're using a fairly recent version of Tunnelblick. At one point a similar problem was resolved when Tunnelblick started clearing the DNS cache on connecting/disconnecting the VPN. You should see a message in the Details… window to that effect. But maybe even that isn't working -- you might want to try stopping and then restarting the mDNSResponder process (my understanding is that it should restart automatically when killed, but…).

On OS X, some commands (I think including ping) use /etc/resolv.conf, and some use scutil. scutil is the "official" data and resolve.conf is supposed to be a read-only interpretation of the data maintained by scutil. My understanding is that sometimes resolve.conf isn't up-to-date. This may be why ping doesn't appear to be working.

[Nicholas]

Thanks for the comments. I have the very latest version of
Tunnelblick. I see the message that the DNS cache is being cleared. As
noted above, both "/etc/resolv.conf" and "scutil --dns" correctly
report that my VPN gateway (172.16.1.1) is the nameserver domain and
search domain are "local," so everything is correct there. I tried
restarting the mDNSResponder process, to no avail. Nothing works. Both
of the "host" and "dig" commands resolve "lynyrd" and "lynyrd.local"
to 192.168.122.101, but no other applications (from ping to ssh to
Safari) can do so. This is very frustrating.

Here is the output of my VPN session (with sensitive information
masked):

2011-04-24 09:18:00 *Tunnelblick: Attempting connection with ___; Set
nameserver = 1; monitoring connection
2011-04-24 09:18:00 *Tunnelblick: /Applications/Tunnelblick.app/
Contents/Resources/openvpnstart start ___.tblk 1337 1 0 0 0 49
2011-04-24 09:18:00 OpenVPN 2.1.4 i386-apple-darwin10.7.1 [SSL] [LZO2]
[PKCS11] built on Mar 1 2011
2011-04-24 09:18:00 MANAGEMENT: TCP Socket listening on 127.0.0.1:1337
2011-04-24 09:18:00 Need hold release from management interface,
waiting...
2011-04-24 09:18:00 MANAGEMENT: Client connected from 127.0.0.1:1337
2011-04-24 09:18:00 MANAGEMENT: CMD 'pid'
2011-04-24 09:18:00 MANAGEMENT: CMD 'state on'
2011-04-24 09:18:00 MANAGEMENT: CMD 'state'
2011-04-24 09:18:00 MANAGEMENT: CMD 'hold release'
2011-04-24 09:18:00 WARNING: No server certificate verification method
has been enabled. See http://openvpn.net/howto.html#mitm for more
info.
2011-04-24 09:18:00 NOTE: the current --script-security setting may
allow this configuration to call user-defined scripts
2011-04-24 09:18:00 Control Channel MTU parms [ L:1541 D:138 EF:38 EB:
0 ET:0 EL:0 ]
2011-04-24 09:18:00 Socket Buffers: R=[42080->65536] S=[9216->65536]
2011-04-24 09:18:00 MANAGEMENT: >STATE:1303654680,RESOLVE,,,
2011-04-24 09:18:00 Data Channel MTU parms [ L:1541 D:1450 EF:41 EB:4
ET:0 EL:0 ]
2011-04-24 09:18:00 Local Options hash (VER=V4): '3514370b'
2011-04-24 09:18:00 Expected Remote Options hash (VER=V4): '239669a8'
2011-04-24 09:18:00 UDPv4 link local: [undef]
2011-04-24 09:18:00 UDPv4 link remote: x.x.x.x:1194
2011-04-24 09:18:00 MANAGEMENT: >STATE:1303654680,WAIT,,,
2011-04-24 09:18:00 MANAGEMENT: >STATE:1303654680,AUTH,,,
2011-04-24 09:18:00 TLS: Initial packet from x.x.x.x:1194,
sid=0e61cef0 c3ee5a8e
2011-04-24 09:18:00 *Tunnelblick: openvpnstart: /Applications/
Tunnelblick.app/Contents/Resources/openvpn --cd /Users/Nicholas/
Library/Application Support/Tunnelblick/Configurations/____.tblk/
Contents/Resources --daemon --management 127.0.0.1 1337 --config /
Users/Nicholas/Library/Application Support/Tunnelblick/Configurations/
____.tblk/Contents/Resources/config.ovpn --log /Library/Application
Support/Tunnelblick/Logs/-SUsers-SNicholas-SLibrary-SApplication
Support-STunnelblick-SConfigurations-S____.tblk-SContents-SResources-
Sconfig.ovpn.1_0_0_0_49.1337.openvpn.log --management-query-passwords
--management-hold --script-security 2 --up /Applications/
Tunnelblick.app/Contents/Resources/client.up.tunnelblick.sh -m -w -d --
down /Applications/Tunnelblick.app/Contents/Resources/
client.down.tunnelblick.sh -m -w -d --up-restart
2011-04-24 09:18:01 VERIFY OK: depth=1, /C=US/ST=TN/L=____/O=NWTS/
CN=_____CA/name=____/emailAddress=noc@____.net
2011-04-24 09:18:01 VERIFY OK: depth=0, /C=US/ST=TN/L=____/O=NWTS/
CN=server/name=____/emailAddress=noc@____.net
2011-04-24 09:18:01 Data Channel Encrypt: Cipher 'BF-CBC' initialized
with 128 bit key
2011-04-24 09:18:01 Data Channel Encrypt: Using 160 bit message hash
'SHA1' for HMAC authentication
2011-04-24 09:18:01 Data Channel Decrypt: Cipher 'BF-CBC' initialized
with 128 bit key
2011-04-24 09:18:01 Data Channel Decrypt: Using 160 bit message hash
'SHA1' for HMAC authentication
2011-04-24 09:18:01 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-
AES256-SHA, 1024 bit RSA
2011-04-24 09:18:01 [server] Peer Connection Initiated with x.x.x.x:
1194
2011-04-24 09:18:02 MANAGEMENT: >STATE:1303654682,GET_CONFIG,,,
2011-04-24 09:18:03 SENT CONTROL [server]: 'PUSH_REQUEST' (status=1)
2011-04-24 09:18:03 PUSH: Received control message: 'PUSH_REPLY,dhcp-
option DNS 172.16.1.1,route 192.168.1.254 255.255.255.0,route
192.168.122.254 255.255.255.0,dhcp-option DOMAIN local,route-gateway
172.16.1.1,topology subnet,ping 10,ping-restart 60,ifconfig 172.16.1.3
255.255.255.0'
2011-04-24 09:18:03 OPTIONS IMPORT: timers and/or timeouts modified
2011-04-24 09:18:03 OPTIONS IMPORT: --ifconfig/up options modified
2011-04-24 09:18:03 OPTIONS IMPORT: route options modified
2011-04-24 09:18:03 OPTIONS IMPORT: route-related options modified
2011-04-24 09:18:03 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option
options modified
2011-04-24 09:18:03 ROUTE default_gateway=10.0.1.1
2011-04-24 09:18:03 TUN/TAP device /dev/tun0 opened
2011-04-24 09:18:03 MANAGEMENT: >STATE:1303654683,ASSIGN_IP,,
172.16.1.3,
2011-04-24 09:18:03 /sbin/ifconfig tun0 delete
ifconfig: ioctl (SIOCDIFADDR):
Can't assign requested address
2011-04-24 09:18:04 NOTE: Tried to delete pre-existing tun/tap
instance -- No Problem if failure
2011-04-24 09:18:04 /sbin/ifconfig tun0 172.16.1.3 172.16.1.3 netmask
255.255.255.0 mtu 1500 up
2011-04-24 09:18:04 /sbin/route add -net 172.16.1.0 172.16.1.3
255.255.255.0
add net 172.16.1.0: gateway
172.16.1.3
2011-04-24 09:18:04 /Applications/Tunnelblick.app/Contents/Resources/
client.up.tunnelblick.sh -m -w -d tun0 1500 1541 172.16.1.3
255.255.255.0 init
No such key
2011-04-24 09:18:04 MANAGEMENT: >STATE:1303654684,ADD_ROUTES,,,
2011-04-24 09:18:04 /sbin/route add -net 192.168.1.254 172.16.1.1
255.255.255.0
add net 192.168.1.254: gateway
172.16.1.1
2011-04-24 09:18:04 /sbin/route add -net 192.168.122.254 172.16.1.1
255.255.255.0
add net 192.168.122.254:
gateway 172.16.1.1
2011-04-24 09:18:04 Initialization Sequence Completed
2011-04-24 09:18:04 MANAGEMENT: >STATE:1303654684,CONNECTED,SUCCESS,
172.16.1.3,x.x.x.x
2011-04-24 09:18:04 *Tunnelblick client.up.tunnelblick.sh: Up to two
'No such key' warnings are normal and may be ignored
2011-04-24 09:18:04 *Tunnelblick client.up.tunnelblick.sh: Saved the
DNS and WINS configurations for later use
2011-04-24 09:18:04 *Tunnelblick client.up.tunnelblick.sh: Set up to
monitor system configuration with leasewatch
2011-04-24 09:18:04 *Tunnelblick: Flushed the DNS cache
2011-04-24 09:18:09 *Tunnelblick leasewatch: A system configuration
change was ignored because it was not relevant
2011-04-24 09:22:20 event_wait : Interrupted system call (code=4)
2011-04-24 09:22:20 TCP/UDP: Closing socket
2011-04-24 09:22:20 /sbin/route delete -net 192.168.122.254 172.16.1.1
255.255.255.0
route: writing to routing
socket: No such process
delete net 192.168.122.254:
gateway 172.16.1.1: not in table
2011-04-24 09:22:20 /sbin/route delete -net 192.168.1.254 172.16.1.1
255.255.255.0
route: writing to routing
socket: No such process
delete net 192.168.1.254:
gateway 172.16.1.1: not in table
2011-04-24 09:22:20 Closing TUN/TAP interface
2011-04-24 09:22:20 /Applications/Tunnelblick.app/Contents/Resources/
client.down.tunnelblick.sh -m -w -d tun0 1500 1541 172.16.1.3
255.255.255.0 init
2011-04-24 09:22:20 SIGTERM[hard,] received, process exiting
2011-04-24 09:22:20 MANAGEMENT: >STATE:1303654940,EXITING,SIGTERM,,
2011-04-24 09:22:20 *Tunnelblick client.down.tunnelblick.sh: Cancelled
monitoring of system configuration changes
2011-04-24 09:22:20 *Tunnelblick client.down.tunnelblick.sh: Restored
the DNS and WINS configurations
2011-04-24 09:22:21 *Tunnelblick: Flushed the DNS cache

[jkbull...gmail.com]

Three things that I see:

1. You are using "Set nameserver". You should try "Set nameserver (alternate 1)" and try removing the "local" domain stuff.

2. There may be some kind of routing issue -- the following messages as the tunnel is torn down are unusual:

2011-04-24 09:22:20

/sbin/route delete -net 192.168.122.254 172.16.1.1 255.255.255.0 

route: writing to routing socket: No such process 

delete net 192.168.122.254: gateway 172.16.1.1: not in table 

2011-04-24 09:22:20

/sbin/route delete -net 192.168.1.254 172.16.1.1 255.255.255.0 

route: writing to routing socket: No such process 

delete net 192.168.1.254: gateway 172.16.1.1: not in table

3. You are specifying "topology subnet". From the OpenVPN man page describing that option:

"When used on *nix, requires that the tun driver supports an ifconfig(8) command which sets a subnet instead of a remote endpoint IP address."

I don't know if the tun driver that Tunnelblick has that support. Tunnelblick's tun/tap drivers are from the tuntap project at http://tuntaposx.sourceforge.net/index.xhtml 

[Nicholas]

Well it took me a LOT of Googling to figure it out, but hopefully this
will help other users, and hopefully it will help you help other users
in the future.

1) First, "Set nameserver (alternate 1)" worked worse for me, because
not only could I still not ping/access the server via the hostname, I
couldn't even resolve it using host/dig/nslookup. With "Set
nameserver" and "Set nameserver (3.0b10)" I could host/dig/nslookup
resolve, but not resolve through ping or other applications.

2) As far as I can tell, there is no routing issue. I could access ALL
servers on any routes that I pushed through their IP addresses. Just
not their hostnames. Definitely a DNS issue, not a routing issue. I'm
not sure what those errors mean, but it appears they weren't the cause
of my problem.

3) Of the two topology options "subnet" and "point-to-point," "subnet"
is the default in OpenVPN if you don't specify otherwise, and it's the
ONLY one that works in Windows. Since my VPN has to support Windows
AND Mac clients, "subnet" is my only choice. Also, I want my clients
to be able to access each other, which only "subnet" allows. I have
not delved into the driver very deep (and I do intend to later), but
it appears it works, because I'm not having any IP issues at all.

Answer) I finally got my hint when I decided maybe the issue wasn't
related to OpenVPN or Tunnelblick at all, and that maybe it was just a
Mac OS X quirk. Searching for 'mac os x ping "unknown host"' on Google
returned pages of results of people able to resolve hosts with host/
dig/nslookup but not ping or access from other apps. Of particular
interest was this link: http://stackoverflow.com/questions/3792272/how-to-ping-xp-machine-from-macosx

OS X computers advertise themselves with multicast DNS (mDNS or
Bonjour as it is often known). host/dig/nslookup use their own
utilities and ALWAYS use only DNS for performing lookups. All other
applications that just use the Mac network services are different.
When no search domain is defined and no FQDN is specified, Mac OS X
appends ".local" to it. When a query ends in ".local" (whether it's
auto appended or the search domain or just part of the query), Mac OS
X will ALWAYS look ONLY in mDNS for a result. It will NOT perform a
DNS query against the nameserver defined. Mac OS X was trying and
failing to resolve lynyrd.local in mDNS and leaving it at that. When I
changed "lynyrd.local" to "lynyrd.nickhq.com" (made-up internal domain
name) both host/dig/nslookup resolution and ping/ssh/other resolution
worked just fine.

In short, don't use ".local" domains, people, and don't fail to
specify a domain (because that will just be ".local," too)! Now I've
done what I should have done in the first place and specified a
private domain name for my entire network, and all DNS resolution is
working perfectly now.

Thanks for the help,

Nick

On Apr 24, 9:53 am, "jkbull...gmail.com" <jkbull...@gmail.com> wrote:
> Three things that I see:
>
> 1. You are using "Set nameserver". You should try "Set nameserver (alternate
> 1)" and try removing the "local" domain stuff.
>
> 2. There may be some kind of routing issue -- the following messages as the
> tunnel is torn down are unusual:
>
> 2011-04-24 09:22:20
>
> /sbin/route delete -net 192.168.122.254 172.16.1.1 255.255.255.0
>
> route: writing to routing socket: No such process
>
> delete net 192.168.122.254: gateway 172.16.1.1: not in table
>
> 2011-04-24 09:22:20
>
> /sbin/route delete -net 192.168.1.254 172.16.1.1 255.255.255.0
>
> route: writing to routing socket: No such process
>
> delete net 192.168.1.254: gateway 172.16.1.1: not in table
>
> 3. You are specifying "topology subnet". From the OpenVPN man page
> describing that option:
>

> "When used on *nix, requires that the tun driver supports an *ifconfig*(8)