x/net/Listen behaviours

165 views
Skip to first unread message

jgr...@ou.edu

unread,
Sep 5, 2019, 4:13:31 PM9/5/19
to golang-nuts
Hi!

I'm having some confusion over the behaviour of net.Listen() and it's interactions with http.Server.

Can anyone take a look at this, and let me know what I'm doing wrong?

Thanks!


System description
-------------------------

Go: go version go1.12.9 darwin/amd64

OS: macOS Mojave (10.14.6)


Problem description
--------------------------

Passing a net.Listener from net.Listen() into the Serve() method of an http.Server does not behave how I expect ...


Test program
-----------------

A simple server that responds to connections by echoing info regarding the URL by which it was contacted (see below):

package main

import (
"context"
"flag"
"fmt"
"log"
"net"
"net/http"
"os"
"os/signal"
"strconv"
"syscall"
"time"
)

// Print information about the local machine's network interfaces
func printNetworkInterfaces() {
ifaces, err := net.Interfaces()
if err != nil { panic("net.Interfaces()") }

if len(ifaces)<1 {
log.Println("No network interfaces found.")
return
}

hostname, _ := os.Hostname()
log.Println( "Network interfaces for " + hostname )

for _, iface := range ifaces {
addrs, err := iface.Addrs()
if err != nil { panic("iface.Addrs()") }
if len(addrs) < 1 { continue }
log.Println("-",iface.Name,iface.HardwareAddr)
for _, addr := range addrs {
switch v := addr.(type) {
case *net.IPNet:
str := fmt.Sprintf("IPNet: IP=%s, mask=%s, network=%s, string=%s", v.IP, v.Mask, v.Network(), v.String())
log.Println(" ", str)
case *net.IPAddr:
str := fmt.Sprintf("IPAddr: IP=%s, zone=%s, network=%s, string=%s", v.IP, v.Zone, v.Network(), v.String())
log.Println(" ", str)
default:
log.Println("<unknown>")
}
}
}
}

// Just write the incoming url back to the sender
func echoHandler(w http.ResponseWriter, r *http.Request) {
txt := fmt.Sprintf("Echo: (%s)",r.URL.Path)
w.Write( []byte(txt+"\n") )
log.Println(txt)
}

var (
listener_ = flag.Int("listener", 0, "Use an explicit net.Listener.")
port_     = flag.Int("port", 0, "Set the port to listen on (0 = any free port?).")
timeout_  = flag.Int("wait", 0, "Timout (in seconds) before server killed (0 = no timout).")
)

func main() {

onShutdown := func(what string, cleanup func()) {
log.Println( fmt.Sprintf("- Shutting down %s ...",what) )
cleanup()
log.Println( fmt.Sprintf("  %s shut down.",what) )
}

flag.Parse()

useListener := *listener_
port := *port_
timeout := *timeout_

// Let's see what interfaces are present on the local machine

printNetworkInterfaces()

// Simple server for incoming connections.

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {echoHandler(w,r)});

var listener net.Listener = nil
addrString := fmt.Sprintf(":%d",port)

// Using an explicit Listener provides more control over the specifics,
// e.g. tcp4/6 and letting the system select a currently free port.

if useListener>0 {
log.Println("Using net.Listener")

listener, err := net.Listen("tcp4", addrString) // :0 -> use any free port
if err != nil { log.Fatalln(err) }

defer onShutdown("listener", func() {listener.Close()} )

addrString = listener.Addr().String()
host, portStr, err := net.SplitHostPort(addrString) // as port may have been assigned by system
if err != nil { log.Fatalln(err) }

log.Println( fmt.Sprintf("Listener Addr string: %s (host: %s, port: %s)",addrString,host,portStr) )

port, err = strconv.Atoi(portStr)
if err != nil { log.Fatalln(err) }

addrString = fmt.Sprintf(":%d",port) // as port may have been assigned by the system
}

server := http.Server { Addr: addrString }

// Run web server in a separate goroutine so it doesn't block our progress

go func(server *http.Server, listener net.Listener) {

var err error

if listener == nil {
err = server.ListenAndServe()
} else {
err = server.Serve(listener)
}

switch err {
case nil:
case http.ErrServerClosed:
log.Println("Caught ErrServerClosed")
default:
panic(err)
}
}(&server, listener)

defer onShutdown("server", func() {server.Shutdown(context.Background())} )

log.Println("Port:", port)
log.Println("Address:", addrString)

// User interrupt channel

sig := make(chan os.Signal, 1)
signal.Notify(sig, os.Interrupt, syscall.SIGTERM)

// Timeout channel, if needed

tc := make(<-chan time.Time);
if timeout > 0 {
tc = time.After(time.Second * time.Duration(timeout))
}

// Wait on user interrupt or timeout

select {
case <-sig: // user interrupt
case <-tc: // timeout
}

// Cleanup

log.Println("Shutting down.")
}

According to the Go docs:

For TCP networks, if the host in the address parameter is empty or a literal unspecified IP address, Listen listens on all available unicast and anycast IP addresses of the local system. To only use IPv4, use network "tcp4".

It also says "See func Dial for a description of the network and address parameters", from which:

Known networks are "tcp", "tcp4" (IPv4-only), "tcp6" (IPv6-only), "udp", "udp4" (IPv4-only), "udp6" (IPv6-only), "ip", "ip4" (IPv4-only), "ip6" (IPv6-only), "unix", "unixgram" and "unixpacket".

Therefore, I would expect that calling net.Listen("tcp4",":0") will listen on an (arbitrary) free port using all IPv4 interfaces. However, lsof indicates that it's listening for both IPv4 and IPv6 (I've masked some identifying information):

me$ go run . --listener=1

2019/09/05 11:14:44 Network interfaces for XXXX
2019/09/05 11:14:44 - lo0 
2019/09/05 11:14:44   IPNet: IP=127.0.0.1, mask=ff000000, network=ip+net, string=127.0.0.1/8
2019/09/05 11:14:44   IPNet: IP=::1, mask=ffffffffffffffffffffffffffffffff, network=ip+net, string=::1/128
2019/09/05 11:14:44   IPNet: IP=fe80::1, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::1/64
2019/09/05 11:14:44 - en0 x:x:x:x:x:x
2019/09/05 11:14:44   IPNet: IP=fe80::x:x:x:x, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::x:x:x:x/64
2019/09/05 11:14:44   IPNet: IP=10.195.66.129, mask=fffff800, network=ip+net, string=10.195.66.129/21
2019/09/05 11:14:44 - utun0 
2019/09/05 11:14:44   IPNet: IP=fe80::x:x:x:x, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::x:x:x:x/64
2019/09/05 11:18:37 Using net.Listener
2019/09/05 11:18:37 Listener Addr string: 0.0.0.0:53133 (host: 0.0.0.0, port: 53133)
2019/09/05 11:14:44 Port: 53133
2019/09/05 11:14:44 Address: :53133

me, in another Terminal window$ lsof -i | grep LISTEN

ARDAgent    360 me    9u  IPv6 0xd5f6fecc3cdf4c41      0t0  TCP *:net-assistant (LISTEN)
LogiVCCor   935 me    9u  IPv4 0xd5f6fecc47403101      0t0  TCP *:iims (LISTEN)
LogiVCCor   935 me   14u  IPv6 0xd5f6fecc3cdf40c1      0t0  TCP *:iims (LISTEN)
go_listen 14859 me    3u  IPv4 0xd5f6fecc451a7781      0t0  TCP *:53133 (LISTEN)
go_listen 14859 me    5u  IPv6 0xd5f6fecc3cdf4681      0t0  TCP *:53133 (LISTEN)

Furthermore, it's only responding on localhost and [::1]; using any of the other interface addresses listed by net.Interfaces() fails to get a response:

me$ time curl localhost:53133/localhost
Echo: (/localhost)

real 0m0.015s
user 0m0.004s
sys 0m0.004s

me$ time curl [::1]:53133/[::1]
Echo: (/[::1])

real 0m0.015s
user 0m0.004s
sys 0m0.005s

^C

real 0m4.521s
user 0m0.004s
sys 0m0.004s

me$ time curl [fe80::1]:53133/[fe80::1]
curl: (7) Couldn't connect to server

real 0m0.014s
user 0m0.004s
sys 0m0.004s

^C

real 0m6.465s
user 0m0.004s
sys 0m0.004s

me$ time curl [fe80::x:x:x:x]:53133/[fe80::x:x:x:x]
curl: (7) Couldn't connect to server

real 0m0.013s
user 0m0.004s
sys 0m0.004s

It looks like trying to connect via (non-localhost) IPv4 addresses hangs on both lo0 and en0 (127.0.0.1, 10.195.66.129), and (non-[::1]) IPv6 addresses flat out refuse to connect.

However, if we skip the use of a net.Listener, it looks like the http.Server only listens for IPv6:

me$ go run .
2019/09/05 11:26:38 Network interfaces for XXXX
2019/09/05 11:26:38 - lo0 
2019/09/05 11:26:38   IPNet: IP=127.0.0.1, mask=ff000000, network=ip+net, string=127.0.0.1/8
2019/09/05 11:26:38   IPNet: IP=::1, mask=ffffffffffffffffffffffffffffffff, network=ip+net, string=::1/128
2019/09/05 11:26:38   IPNet: IP=fe80::1, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::1/64
2019/09/05 11:26:38 - en0 x:x:x:x:x:x
2019/09/05 11:26:38   IPNet: IP=fe80::x:x:x:x, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::x:x:x:x/64
2019/09/05 11:26:38   IPNet: IP=10.195.66.129, mask=fffff800, network=ip+net, string=10.195.66.129/21
2019/09/05 11:26:38 - utun0 
2019/09/05 11:26:38   IPNet: IP=fe80::x:x:x:x, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::x:x:x:x/64
2019/09/05 11:26:38 Port: 0
2019/09/05 11:26:38 Address: :0

me, in another Terminal window$ lsof -i | grep LISTEN
ARDAgent    360 me    9u  IPv6 0xd5f6fecc3cdf4c41      0t0  TCP *:net-assistant (LISTEN)
LogiVCCor   935 me    9u  IPv4 0xd5f6fecc47403101      0t0  TCP *:iims (LISTEN)
LogiVCCor   935 me   14u  IPv6 0xd5f6fecc3cdf40c1      0t0  TCP *:iims (LISTEN)
go_listen 15016 me    3u  IPv6 0xd5f6fecc3cdf5201      0t0  TCP *:53170 (LISTEN)

This approach (i.e., ignoring net.Listener to simply use server.ListenAndServe()) seems to do a better job of listening on multiple interfaces/addresses:

me$ time curl localhost:53170/localhost
Echo: (/localhost)

real 0m0.015s
user 0m0.004s
sys 0m0.004s

me$ time curl [::1]:53170/[::1]
Echo: (/[::1])

real 0m0.015s
user 0m0.004s
sys 0m0.004s

Echo: (/127.0.0.1)

real 0m0.014s
user 0m0.004s
sys 0m0.004s

me$ time curl [fe80::1]:53170/[fe80::1]
curl: (7) Couldn't connect to server

real 0m0.014s
user 0m0.004s
sys 0m0.004s

Echo: (/10.195.66.129)

real 0m0.015s
user 0m0.004s
sys 0m0.004s

me$ time curl [fe80::x:x:x:x]:53170/[fe80::x:x:x:x]
curl: (7) Couldn't connect to server

real 0m0.015s
user 0m0.004s
sys 0m0.004s

... although it also has problems for IPv6 addresses other than ::1.

If I use tcp instead of tcp4 in the call to net.Listen() (e.g. net.Listen("tcp", ":0")) I always get a bind error due to address already in use:

me$ go run . --listener=1
2019/09/05 11:45:13 Network interfaces for XXXX
2019/09/05 11:45:13 - lo0 
2019/09/05 11:45:13   IPNet: IP=127.0.0.1, mask=ff000000, network=ip+net, string=127.0.0.1/8
2019/09/05 11:45:13   IPNet: IP=::1, mask=ffffffffffffffffffffffffffffffff, network=ip+net, string=::1/128
2019/09/05 11:45:13   IPNet: IP=fe80::1, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::1/64
2019/09/05 11:45:13 - en0 x:x:x:x:x:x
2019/09/05 11:45:13   IPNet: IP=fe80::x:x:x:x, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::x:x:x:x/64
2019/09/05 11:45:13   IPNet: IP=10.195.66.129, mask=fffff800, network=ip+net, string=10.195.66.129/21
2019/09/05 11:45:13 - utun0 
2019/09/05 11:45:13   IPNet: IP=fe80::x:x:x:x, mask=ffffffffffffffff0000000000000000, network=ip+net, string=fe80::x:x:x:x/64
2019/09/05 11:45:13 Using net.Listener
2019/09/05 11:45:13 Listener Addr string: [::]:53260 (host: ::, port: 53260)
2019/09/05 11:45:13 Port: 53260
2019/09/05 11:45:13 Address: :53260
panic: listen tcp :53260: bind: address already in use

I get the same outcome if I use tcp6 in net.Listen(); I can only get the program to run using tcp4 which, on my machine at least, actually seems to open an additional IPv6 connection anyway - so I'm confused as to why net.Listen() doesn't seem to like tcp6.

Despite the net.Listen() documentation directing you to net.Dial() docs for a discussion of the network parameter, some of these networks (e.g. ip, ip4, ip6) are unknown to net.Listen(). The documentation could be clearer in that respect! :)

I can't figure out what's going on here. Any ideas what I might be doing wrong?

Jacques Supcik

unread,
Sep 6, 2019, 9:09:37 AM9/6/19
to golang-nuts
Hello,

I tried your program and when you call listener, err := net.Listen("tcp4", addrString) it does indeed only listen to "tcp4". Later, when you call server.ListenAndServe() (because listener is nil), the system seems to "re-listen" from the same port and then it listen to "tcp" and not only "tcp4" (https://github.com/golang/go/blob/master/src/net/http/server.go#L2826)

I don't understand why you call both "net.Listen" and "server.ListenAndServe". If you want an HTTP server, I think that you can just use "server.ListenAndServe" and you don't need anything more. If you want an HTTP server and a generic TCP server, then you need 2 different ports, one for the HTTP server and another for the generic TCP server. But perhaps I did not understand your problem ;-)

-- Jacques

jgr...@ou.edu

unread,
Sep 6, 2019, 9:43:21 AM9/6/19
to golang-nuts
Hi Jacques,

Thanks for the help!


> when you call listener, err := net.Listen("tcp4", addrString) it does indeed only listen to "tcp4". Later, when you call server.ListenAndServe() (because listener is nil), the system seems to "re-listen" from the same port and then it listen to "tcp" and not only "tcp4" (https://github.com/golang/go/blob/master/src/net/http/server.go#L2826)

I only call server.ListenAndServe() if the listener object is nil; therefore, ListenAndServe() should not be called after listener, err := net.Listen("tcp4", addrString)?

The listener is nil by default, unless specified otherwise on the command line (e.g., --listener=1); at that point. it is initialised (listener, err := net.Listen("tcp4", addrString)) and the server object should then use server.Serve(listener) rather than server.ListenAndServe().

Sorry for the lack of code clarity - and thanks again for the help, I appreciate it!

Cheers,

J.

jgr...@ou.edu

unread,
Sep 6, 2019, 9:52:11 AM9/6/19
to golang-nuts
Ahh, I think I understand the problem!

I assumed that "listener, err := ..." would use the existing listener variable (creating err on the spot), rather than creating a new locally-scoped listener?

Therefore, as you said, listener is basically always nil outside that "if useListener>0 {}" scope? So server.ListenAndServe() is always invoked?

If so, then - thanks a lot for the help! :)

Cheers,

J.

On Friday, September 6, 2019 at 8:09:37 AM UTC-5, Jacques Supcik wrote:

Jacques Supcik

unread,
Sep 6, 2019, 11:00:13 AM9/6/19
to golang-nuts
Yes! I think that your analysis is correct. The listener inside the if is not the same as the one outside. You can check using fmt.Println(&listener) and you will see two different addresses.

-- Jacques

jgr...@ou.edu

unread,
Sep 6, 2019, 1:06:19 PM9/6/19
to golang-nuts
You're absolutely right, Jacques!

I completely misunderstood this from the Go documentation:

Redeclaration does not introduce a new variable; it just assigns a new value to the original.

.. because I did not understand the difference between the scope of a variable and the block in which it was declared.

Thanks again for the help!

J.

joe mcguckin

unread,
Sep 18, 2019, 8:06:11 PM9/18/19
to golang-nuts
Is listener() a blocking operation?
Reply all
Reply to author
Forward
0 new messages