Problem with IPv6-only host and DNS resolution order

[falzo...@gmail.com]

Hi there

I'm facing an odd issue with Go's net package: it looks like Go is systematically preferring IPv4 over IPv6 (even bypassing the OS settings, for instance from /etc/gai.conf) when trying to net.Dial() a host with dual A/AAAA DNS records, thus panic()ing with a "network unreachable" error when trying to connect to it because the running host only has IPv6 connectivity.

Note: all tests have been performed with host www.kame.net but are reproducible with www.google.com, www.golang.org and any dual IPv4/IPv6 hostname.

root@xxx:~# ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:16:3e:e7:8f:a9 brd ff:ff:ff:ff:ff:ff

    inet6 2001:4b98:dc0:xxxx:xxxx:xxxx:xxxx:xxxx/64 scope global dynamic 

       valid_lft 2591947sec preferred_lft 604747sec

    inet6 fe80::216:xxxx:xxx:xxx/64 scope link 

       valid_lft forever preferred_lft forever

root@xxx:~# dig +short A www.kame.net

orange.kame.net.

203.178.141.194

root@xxx:~# dig +short AAAA www.kame.net

orange.kame.net.

2001:200:dff:fff1:216:3eff:feb1:44d7

DNS resolution works as intended (i.e. IPv6 either first or only choice) with utilities such as wget, curl and getent:

root@xxx:~# getent hosts www.kame.net

2001:200:dff:fff1:216:3eff:feb1:44d7 orange.kame.net www.kame.net

root@xxx:~# curl -I http://www.kame.net

HTTP/1.1 200 OK

Date: Fri, 02 May 2014 15:25:35 GMT

Server: Apache/2.2.26 (FreeBSD) mod_ssl/2.2.26 OpenSSL/0.9.8y DAV/2

Accept-Ranges: bytes

Content-Type: text/html

root@xxx:~# curl -I -6 http://www.kame.net

HTTP/1.1 200 OK

Date: Fri, 02 May 2014 15:25:23 GMT

Server: Apache/2.2.26 (FreeBSD) mod_ssl/2.2.26 OpenSSL/0.9.8y DAV/2

Accept-Ranges: bytes

Content-Type: text/html

root@xxx:~# curl -I -4 http://www.kame.net

curl: (7) Failed to connect to 203.178.141.194: Network is unreachable

root@xxx:~# wget --spider http://www.kame.net

Spider mode enabled. Check if remote file exists.

--2014-05-02 17:27:25--  http://www.kame.net/

Resolving www.kame.net (www.kame.net)... 2001:200:dff:fff1:216:3eff:feb1:44d7, 203.178.141.194

Connecting to www.kame.net (www.kame.net)|2001:200:dff:fff1:216:3eff:feb1:44d7|:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: unspecified [text/html]

Remote file exists and could contain further links,

but recursion is disabled -- not retrieving.

root@xxx:~# wget -6 --spider http://www.kame.net

Spider mode enabled. Check if remote file exists.

--2014-05-02 17:27:46--  http://www.kame.net/

Resolving www.kame.net (www.kame.net)... 2001:200:dff:fff1:216:3eff:feb1:44d7

Connecting to www.kame.net (www.kame.net)|2001:200:dff:fff1:216:3eff:feb1:44d7|:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: unspecified [text/html]

Remote file exists and could contain further links,

but recursion is disabled -- not retrieving.

root@xxx:~# wget -4 --spider http://www.kame.net

Spider mode enabled. Check if remote file exists.

--2014-05-02 17:27:49--  http://www.kame.net/

Resolving www.kame.net (www.kame.net)... 203.178.141.194

Connecting to www.kame.net (www.kame.net)|203.178.141.194|:80... failed: Network is unreachable.

root@xxx:~# ping6 -c 3 www.kame.net

PING www.kame.net(2001:200:dff:fff1:216:3eff:feb1:44d7) 56 data bytes

64 bytes from 2001:200:dff:fff1:216:3eff:feb1:44d7: icmp_seq=1 ttl=51 time=267 ms

64 bytes from 2001:200:dff:fff1:216:3eff:feb1:44d7: icmp_seq=2 ttl=51 time=274 ms

64 bytes from 2001:200:dff:fff1:216:3eff:feb1:44d7: icmp_seq=3 ttl=51 time=283 ms

--- www.kame.net ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2002ms

rtt min/avg/max/mdev = 267.961/275.470/283.507/6.386 ms

root@xxx:~# ping -c 3 www.kame.net

connect: Network is unreachable

Now see Go's behaviour:

root@xxx:~# cat test.go 

package main

import "fmt"

import "net"

func main() {

  host := "www.kame.net"

  addrs, err := net.LookupHost(host)

  if err != nil {

    fmt.Printf("net.LookupHost(): error: %s\n", err)

    return

  }

  fmt.Printf("addrs = %s\n", addrs)

  conn, err := net.Dial("tcp", host + ":80")

  if err != nil {

    fmt.Printf("net.Dial(): error: %s\n", err)

    return

  }

  fmt.Printf("conn.RemoteAddr().String() = %s\n", conn.RemoteAddr().String())

}

root@xxx:~# go run test.go

addrs = [2001:200:dff:fff1:216:3eff:feb1:44d7 203.178.141.194]

net.Dial(): error: dial tcp 203.178.141.194:80: network is unreachable

Oddly enough, netcat is also affected in the same way:

root@xxx:~# netcat www.kame.net 80

orange.kame.net [203.178.141.194] 80 (http) : Network is unreachable

Here are some details on my test platform (tested with the same behaviour on Mac OS X), let my know if you need more:

root@xxx:~# go version

go version go1.2 linux/amd64

root@xxx:~# uname -a

Linux xxx 3.2.53-xenU-8869-x86_64 #4 SMP Fri Dec 20 13:49:31 UTC 2013 x86_64 GNU/Linux

root@xxx:~# lsb_release -a

No LSB modules are available.

Distributor ID: Debian

Description:    Debian GNU/Linux 7.5 (wheezy)

Release:    7.5

Codename:   wheezy

I'm fairly confident in my DNS/network setup, but if you need further tests or info let me also know.

Cheers,

Marc.

[Mikio Hara]

try http://play.golang.org/p/bh8AL5TjJY, and pick your favorite way.

# okay, i'm gonna go with California chrome this year. Mint julep!

[Mikio Hara]

On Sat, May 3, 2014 at 11:27 AM, Mikio Hara <mikioh...@gmail.com> wrote:

> try http://play.golang.org/p/bh8AL5TjJY, and pick your favorite way.

fwiw, in the case of http over tls over tcp: http://play.golang.org/p/urfRkZDDpY

[falzo...@gmail.com]

Hi

Yep, definitely works better!

root@xxx:~# go run test_mikio.go

2014/05/03 10:11:43 connected to [2001:200:dff:fff1:216:3eff:feb1:44d7]:80 via [2001:4b98:dc0:51:216:3eff:fee7:8fa9]:35411

2014/05/03 10:11:43 oops dial tcp 203.178.141.194:80: network is unreachable

2014/05/03 10:11:43 connected to [2001:200:dff:fff1:216:3eff:feb1:44d7]:80 via [2001:4b98:dc0:51:216:3eff:fee7:8fa9]:35412

2014/05/03 10:11:43 connected to [2a00:1450:4013:c01::69]:80 via [2001:4b98:dc0:51:216:3eff:fee7:8fa9]:49263

2014/05/03 10:11:43 oops dial tcp 74.125.136.99:80: network is unreachable

2014/05/03 10:11:43 oops dial tcp 74.125.136.103:80: network is unreachable

2014/05/03 10:11:43 oops dial tcp 74.125.136.104:80: network is unreachable

2014/05/03 10:11:43 oops dial tcp 74.125.136.105:80: network is unreachable

2014/05/03 10:11:43 oops dial tcp 74.125.136.106:80: network is unreachable

2014/05/03 10:11:43 oops dial tcp 74.125.136.147:80: network is unreachable

2014/05/03 10:11:44 connected to [2a00:1450:4013:c01::69]:80 via [2001:4b98:dc0:51:216:3eff:fee7:8fa9]:49264

2014/05/03 10:11:44 connected to [2a00:1450:400c:c05::8d]:80 via [2001:4b98:dc0:51:216:3eff:fee7:8fa9]:53026

2014/05/03 10:11:44 oops dial tcp 173.194.67.141:80: network is unreachable

2014/05/03 10:11:44 connected to [2a00:1450:400c:c05::8d]:80 via [2001:4b98:dc0:51:216:3eff:fee7:8fa9]:53027

Originally I intended to do HTTP (non-secured) polling, so in the end it looks like I'll have to go with something like this:

package main

import "fmt"

import "log"

import "net"

import "net/http"

func main() {

        host := "www.kame.net"

        tr := &http.Transport{Dial: (&net.Dialer{DualStack: true}).Dial}

        client := http.Client{Transport: tr}

        resp, err := client.Get(fmt.Sprintf("http://%s/", net.JoinHostPort(host, "80")))

        if err != nil {

                log.Println("oops", err)

                return

        }

        log.Printf("%s: %s %s\n", resp.Request.URL.Host, resp.Proto, resp.Status)

        resp.Body.Close()

}

root@poll1:~# go run test_http.go

2014/05/03 10:22:51 www.kame.net:80: HTTP/1.1 200 OK

Thank you for the tip!

m.