when EOF returned from http client.Post, client can continue to be used ?

[Toon Knapen]

It is not clear to me from the documentation whether I can continue to use an http.Client after a POST failed (due to the connection being closed ;  In my case the connection closes probably due to rate-limiting).

The documentation of the http.Transport does mention that, if a network error occurs, it will retry if the request is idempotent. Given I'm doing a basicPOST that is not the case for me. 

But the http.Client will reestablish the connection on the next call to Post(...) ?

toon

 

[Amit Saha]

I believe you will see that's the case. Perhaps worth experimenting with it? The reason is due to the underlying connection pooling behavior of the  DefaultTransport: https://go.dev/src/net/http/transport.go#L42:

" It establishes network connections as needed // and caches them for reuse by subsequent calls"

toon

 

[Toon Knapen]

I did experiment but when using the same client I immediately got an EOF again. But that might also have been due to the rate-limiting. 

I started reading the source code of the http.Client and DefaultTransport but if any could advice good articles/books about the internals to get the most out of it, I would be very interested.

[Konstantin Khomoutov]

It won't be http.Client who will reestablish the connection, but whatever
implementation supporting the http.RoundTripper interface provided in the
Transport field of the http.Client. If that field is not set, the default
http.DefaultTransport will be used, which is an instance of http.Transport.

Basically the idea is that an http.Transport first tries to pick a live but
idle TCP connection from its pool of idle connections, and if there are none,
it tries to establish a new one.

I'd undeline the idea is that an http.Client is not something akin to a single
invocation of `curl`; instead it's a shared client-side HTTP request
"multiplexer" which is OK to be used for multiple concurrent HTTP requests
wihch will share a single http.RoundTripper configured to be used for that
http.Client (as explained above), and it, in turn, is responsible for
maintaining a pool of ready-to-use connections, any number of which can be
used to carry out HTTP requests concurrently.

[Amit Saha]

On Monday, April 18, 2022 at 6:17:30 PM UTC+10 toon....@gmail.com wrote:

I did experiment but when using the same client I immediately got an EOF again. But that might also have been due to the rate-limiting. 

That's interesting. Could you try printing the response status or writing client middleware to further poke into the internals?

I started reading the source code of the http.Client and DefaultTransport but if any could advice good articles/books about the internals to get the most out of it, I would be very interested.

You can search online for "Go roundtripper" and use them for some experimentation.

My book talks about the transport and connection pooling. Some free preview available: https://books.google.com.au/books?id=T5FNEAAAQBAJ&lpg=PT115&pg=PT137#v=onepage&q&f=false