http.ListenAndServe holding on to port after Process.Kill()

[Mike Murray]

I'm trying to write a Go program that basically wraps a Go HTTP server. The wrapper compiles and runs the HTTP server program, then watches the filesystem and recompiles the HTTP server whenever something changes.

The problem I'm seeing, is that the second time the HTTP server is run it always fails due to the port already being in use.

I start the program using exec.Command like this:

cmd = exec.Command("go", "run", "server.go")

And I later kill it using Process.Kill() like this:

cmd.Process.Kill()

But even if I wait 20 seconds after the Kill() call before starting up the server again, it fails with "address already in use".

Why doesn't Process.Kill() release the port? Is there a way I can release the port?

I pasted the code here for more context: https://gist.github.com/6b3f64f1bc32674a7f67

[Miki Tebeka]

I'm not an expert, but sometimes when a process is killed and the socket is not closed cleanly - the OS doesn't free the port right away.
Maybe there's an option to use SO_REUSEADDR?

Can you post the code for server.go as well?

[Mike Murray]

The code for server.go basically just uses http.ListenAndServe(), so it looks something like:

    http.HandleFunc("/hello", HelloServer)

    err := http.ListenAndServe(":5000", nil)

    if err != nil {

      log.Fatal("ListenAndServe: ", err)

    }

I'll look into SO_REUSEADDR, thanks for the pointer.

[Brad Fitzpatrick]

I doubt it's SO_REUSEADDR.  That's already used.

Are you starting a child process from your webserver process?  It's possible you're leaking the listening fd (the port 5000) to the child, due to a bug which was fixed recently.

[Mike Murray]

No I'm not. My server.go was kind of convoluted so I made a simple example.

This is the server code: https://gist.github.com/e318b4d04809fd6547b1

This is the wrapper code: https://gist.github.com/8ebbca82e01dcbaf1b1f

When I run that on my macbook and then save server.go (to trigger the fsnotify event) this is the output:

2012/08/22 17:48:42 Listening on port 5001...

2012/08/22 17:48:50 event: "server.go": MODIFY

2012/08/22 17:48:56 Listening on port 5001...

2012/08/22 17:48:56 err: listen tcp <nil>:5001: address already in use

[Dave Cheney]

Are you able to invoke netstat immediately after the failure and see who is holding the port open?

[Andy Balholm]

On Wednesday, August 22, 2012 5:56:25 PM UTC-7, Mike Murray wrote:

When I run that on my macbook and then save server.go (to trigger the fsnotify event) this is the output:

I wondered if you were on a Mac. I've run into that problem before; it's not Go-specific. Sometimes when a process ends, its listening socket is not freed instantly. It's usually available a while later (less than a minute anyway). I don't know what's really happening or why, though. 

[Jesse McNelis]

On Thu, Aug 23, 2012 at 10:56 AM, Mike Murray <mmurr...@gmail.com> wrote:
> No I'm not. My server.go was kind of convoluted so I made a simple example.
>
> This is the server code: https://gist.github.com/e318b4d04809fd6547b1
> This is the wrapper code: https://gist.github.com/8ebbca82e01dcbaf1b1f

Doesn't 'go run' fork a separate process to run the compiled code?
Thus, you're killing 'go run' but not it's child process? and not
giving 'go run' a chance to clean up.

Perhaps you want to sigterm it instead.

--
=====================
http://jessta.id.au

[andrey mirtchovski]

Jesse is right. When you kill the 'go' command you don't kill the child. Observe on OSX:

$ go run server.go &

[1] 17246

2012/08/22 19:28:02 Listening on port 5001...

$ killall go

[1]+  Exit 2                  go run server.go

$ ps auwx | grep a.out

andrey         17408   0.0  0.1 20299256   3136 s004  S     7:28PM   0:00.01 /var/folders/sr/t9dx6fjs59gff5hs0p0zl9d00000gm/T/go-build297803053/command-line-arguments/_obj/a.out

$ ./server 

2012/08/22 19:28:27 Listening on port 5001...

2012/08/22 19:28:27 err: listen tcp <nil>:5001: address already in use

$ killall a.out

$ ./server 

2012/08/22 19:29:56 Listening on port 5001...

So don't do 'go run', instead do 'go build' and, if that is successful, run the resulting binary which you can signal directly.

[Mike Murray]

Ah, yep, that did the trick! Thanks for the help!

[woonk...@gmail.com]

Coming to see in May 2019. Thank you for your help very much!

2012년 8월 23일 목요일 오전 10시 40분 53초 UTC+9, andrey mirtchovski 님의 말: