"can't assign requested address" error when doing many concurrent requests using net/http client.

[kalp...@gmail.com]

I'm trying to benchmark a simple hello world server on my computer (osx).

full error: "Get http://localhost:8080/: dial tcp 127.0.0.1:8080: can't assign requested address"

server.go

package main

import (

"fmt"

"net/http"

)

func handler(w http.ResponseWriter, r *http.Request) {

fmt.Fprint(w, "Hello, world")

}

func main() {

http.HandleFunc("/", handler)

http.ListenAndServe(":8080", nil)

}

bench.go

package main

import (

"fmt"

"io"

"io/ioutil"

"net/http"

"sync"

"sync/atomic"

"time"

)

var (

success, fail int64

wg            sync.WaitGroup

)

const (

goroutines = 4

duration   = 30 * time.Second

)

func main() {

timer := time.NewTimer(duration)

for {

select {

case <-timer.C:

fmt.Println("success", success, "fail", fail)

return

default:

wg.Add(goroutines)

for i := 0; i < goroutines; i++ {

go hit("http://localhost:8080/")

}

wg.Wait()

}

}

}

func hit(url string) {

defer wg.Done()

resp, err := http.Get(url) // error occurs here

if err != nil {

fmt.Println(err)

atomic.AddInt64(&fail, 1)

return

}

_, err = io.Copy(ioutil.Discard, resp.Body)

if err != nil {

fmt.Println(err)

}

err = resp.Body.Close()

if err != nil {

fmt.Println(err)

}

if resp.StatusCode != http.StatusOK {

atomic.AddInt64(&fail, 1)

return

}

atomic.AddInt64(&success, 1)

}

I've been googling this problem for a while, but I'm stumped. I think its due to resources not being released (sockets?). Some people had the same error, but it was because they weren't reading and closing the response body, yet I'm doing that and I still get the error.

[Dave Cheney]

Have you increased the number of file descriptors available to your program? By default osx only gives your process 256, some of which are already used by the operating system

[Benjamin Measures]

On Thursday, 26 February 2015 07:37:48 UTC, kalp...@gmail.com wrote:

full error: "Get http://localhost:8080/: dial tcp 127.0.0.1:8080: can't assign requested address"

bench.go

 for {
   // [...]


   wg.Add(goroutines)
   for i := 0; i < goroutines; i++ {
     go hit("http://localhost:8080/")
   }
   wg.Wait()


   // [...]
 }

Ephemeral ports. It's a phrase that comes up often in conjuction with "concurrent" and "client requests".

Your main loop is creating and tearing down `goroutine` connections at a time. Thus, there are `goroutine - DefaultMaxIdleConnsPerHost` connections being closed each outer loop. If each outer loop takes 1ms, you'll run out of ephemeral ports in ~8s.

[Benjamin Measures]

Some code I put together a while ago for something I had to do quickly: https://play.golang.org/p/cWcUA5taI3

By having a connection pool per worker, it uses exactly 100 connections (where directed at a single host).

[kalp...@gmail.com]

That's it, I was running out of ports. I gave each connection its own client and it works perfectly now. Thanks!