http Transport dies too slowly

[Tamás Gulácsi]

Just reuse the created http.Client.

[Amnon Baron Cohen]

As Tamas said, reuse the client.

Just make the client into a global variable.

On Sunday, 27 May 2018 11:29:47 UTC+1, Vadim Lesich wrote:

Do you have an example?

воскресенье, 27 мая 2018 г., 8:39:50 UTC+3 пользователь Tamás Gulácsi написал:

Just reuse the created http.Client.

[Tamás Gulácsi]

var clients = make(map[string]*http.Client)

func checkProxySOCKS(prox string, c chan QR, wg *sync.WaitGroup) (err error) {

defer wg.Done()

httpClient := clients[prox]
if httpClient == nil{
dialer, err := proxy.SOCKS5("tcp", prox, nil, proxy.Direct)
if err != nil {
return
}

timeout := time.Duration(1 * time.Second)

httpClient = &http.Client{
Timeout: timeout,
Transport: &http.Transport{
DisableKeepAlives: true,
Dial: dialer.Dial,
},
}
clients[prox]=httpClient
}

...

[Tamás Gulácsi]

Map is not concurrency safe, you have to synchronize access. For example with a sync.Mutex.

[Amnon Baron Cohen]

No, just get rid of the clients map, and instead, just create a single global client.

Http Clients are designed to be used by multiple goroutines.

On Sun, May 27, 2018 at 4:33 PM, Tamás Gulácsi <tgula...@gmail.com> wrote:

Map is not concurrency safe, you have to synchronize access. For example with a sync.Mutex.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/uDOBUBNIz3o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tamás Gulácsi]

"By default, Transport caches connections for future re-use. This may leave many open connections when accessing many hosts. This behavior can be managed using Transport's CloseIdleConnections method and the MaxIdleConnsPerHost and DisableKeepAlives fields. "

So you may try to close CloseIdleConnections.

2018. május 27., vasárnap 19:26:19 UTC+2 időpontban Vadim Lesich a következőt írta:

Ok with Mutex all work okay but i'm still have unclosedTransports, i'v check it with pprof and he show that

Dial: dialer.Dial,

blocked Transport from closing

воскресенье, 27 мая 2018 г., 18:33:12 UTC+3 пользователь Tamás Gulácsi написал:

[Tamás Gulácsi]

Copy golang.org/x/net/internal/socks (like github.com/tgulacsi/go/x-net-internal-socks) and use socks.NewDialer(...).DialContext instead of Dial.

2018. május 27., vasárnap 19:45:36 UTC+2 időpontban Vadim Lesich a következőt írta:

I got suggestion use DialContext but proxy package not suport it, also i found this https://go-review.googlesource.com/c/net/+/37641

воскресенье, 27 мая 2018 г., 20:26:19 UTC+3 пользователь Vadim Lesich написал:

Ok with Mutex all work okay but i'm still have unclosedTransports, i'v check it with pprof and he show that

Dial: dialer.Dial,

blocked Transport from closing

воскресенье, 27 мая 2018 г., 18:33:12 UTC+3 пользователь Tamás Gulácsi написал:

Map is not concurrency safe, you have to synchronize access. For example with a sync.Mutex.