HTTP Client Timeout

[Ian Davis]

Hi all,

Can someone clarify the behaviour of http.Client's Timeout field for me?
When I've used it in my code then I've found that the timeout is
absolute and applies across all requests made by that client. The
comment on the field suggests this is the case:

// Timeout specifies a time limit for requests made by this
// Client. The timeout includes connection time, any
// redirects, and reading the response body. The timer remains
// running after Get, Head, Post, or Do return and will
// interrupt reading of the Response.Body.

That wording also talks about request in the singular, i.e. "the"
response body. However in the comment for the Client struct itself, it's
suggested that clients should be reused for multiple requests:

The Client's Transport typically has internal state (cached TCP
connections), so Clients should be reused instead of created as
needed.

My reading of the stdlib code suggests that the timeout behaves as
though it we an absolute one across all requests made by the client but
this looks almost accidental because in Client.doFollowingRedirects a
new timer is started for every request (actually an AfterFunc) and no
attempt is made to stop this between requests. For an absolute timeout
I'd expect it to use sync.Once to start the timer on the first request,
so I'm not sure what the intended behaviour is here.

Maybe I am missing something obvious. If not, then perhaps the
documentation needs to be clarified by rewording to something like
"Timeout specifies an overall time limit for all requests made by this
Client."

Cheers,

Ian

[James Bardin]

The timeout is per each individual request, i.e. the client issues a CancelRequest after the timeout period. 

The client itself is intended to be allocated once, and is safe for concurrent use. 

Though, I'm not sure what part of  the docs your misinterpreting, or could be clarified.

[Ian Davis]

 

On Tue, Sep 22, 2015, at 02:05 PM, James Bardin wrote:

The timeout is per each individual request, i.e. the client issues a CancelRequest after the timeout period. 

The client itself is intended to be allocated once, and is safe for concurrent use. 

 

OK thanks.. Re-reading the stdlib makes it clear that CancelRequest is called on each separate request, so I must have an error in my client code. Sorry for the noise.

 

Ian

[James Bardin]

On Tue, Sep 22, 2015 at 9:42 AM, Ian Davis <m...@iandavis.com> wrote:
>
> OK thanks.. Re-reading the stdlib makes it clear that CancelRequest is
> called on each separate request, so I must have an error in my client code.
> Sorry for the noise.

If it helps, an error I've seen a couple times with Client.Timeout is
the reuse of *http.Requests.

The Transport indexes the Requests by their pointers, so reusing the
same object for concurrent calls can result in the wrong request being
canceled.

[Ian Davis]

Thanks. That's a good point. I checked and we're using http.NewRequest
for each request so I think my particular problem must be something
else.

Ian

[Brad Fitzpatrick]

Maybe you were confusing it with the net.Dialer's Deadline and mixing the two somewhere?

In general Go tries to be consistent with terminology:

A "timeout" is relative, and of type time.Duration.

A "deadline" is absolute, and of type time.Time.