context cancellation flake

[kyn...@gmail.com]

Hello,

I was wondering if anyone could help me with understanding the behaviour of cancelling context on a request before its read.

I am using golang 1.9 on macOS X sierra (latest)

Sometimes this code returns context cancelled, sometimes it works.

package main

import (
    "context"
    "io/ioutil"
    "log"
    "net/http"
    "time"

    "golang.org/x/net/context/ctxhttp"
)

func main() {
    req, err := http.NewRequest("GET", "https://swapi.co/api/people/1", nil)
    if err != nil {
        log.Fatal(err)
    }
    resp, err := fetch(req)
    if err != nil {
        log.Fatal(err)
    }
    log.Print(readBody(resp))
}

func fetch(req *http.Request) (*http.Response, error) {
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()
    return ctxhttp.Do(ctx, http.DefaultClient, req)
}

func readBody(resp *http.Response) (string, error) {
    b, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return "", err
    }
    return string(b), err
}

[Tamás Gulácsi]

You "readBody" after "defer cancel()", so response length amd caching may provide you a body, maybe just a canceled context.

[Steven Lee]

Does calling the cancel func on a context that is attached to a http.Request change the contents of the body to context cancellation?

[Tamás Gulácsi]

No, but may cancel the reading before completion.

[Steven Lee]

I guess thats plausible

Ive tried to find the code in the library that sets the body if a context is cancelled but cant, I understand how to fix the code just struggling to understand the mechanics and especially why it flakes :(

[silviu...@gmail.com]

Hi Steven,

In case it's still unclear, you need to wrap both the "do request" + "read body" inside the same cancellation context. The "defer cancel" should encompass both of them, sort of atomically, so the idea is to take it out of your fetch, one level up.

https://play.golang.org/p/trMP7Q-maT

Cheers,

silviu

[Steven Lee]

Thank you Silviu, that seems to work.

Do you know exactly why it flakes? what is racing? just for me to have an understanding of why this happens

[silviu...@gmail.com]

Steven, no problem. For more details have a look at

a) This talk by Sameer Ajmani: https://vimeo.com/115309491

b) Francesc Campoy's series of videos: https://github.com/campoy/justforfunc  

In the Readme, you'll find likes to episodes 9 and 10, which deal with explaning and implementing the context package.

But, to answer your question very quickly: the purpose for the cancel function is to voluntarily signal to the rest of your program that one particular flow of operations (determined by you, when you design your application) is over.

You are calling it as a courtesy to some other go routines inside your program which may depend on, or act upon the same operation.

In your case, you want to read from that external endpoint some json or xml values, right ? 

As far as you're concerned, the real "finish line" of that particular flow is after ioutil.ReadAll, not the ctxhttp.Do method, correct ? That's why you need to make sure you call cancel() after that one, not too early.

Due to the fact that you are calling cancel too early, as Tamás pointed out, you are not guaranteed to receive the response body, because the http internal mechanisms are not synchronous and are complex enough.

If you have some good hours to spend (take your time, to understand it correctly), dig into the source code directly. Start with 

https://github.com/golang/net/blob/master/context/ctxhttp/ctxhttp.go#L26

that will take you into:

https://github.com/golang/go/blob/master/src/net/http/client.go#L490

and so forth...

[Jesper Louis Andersen]

On Sun, Sep 17, 2017 at 12:46 PM Steven Lee <kyn...@gmail.com> wrote:

Thank you Silviu, that seems to work.

Do you know exactly why it flakes? what is racing? just for me to have an understanding of why this happens

It is somewhat common to use the TCP window as a feedback mechanism to the server. If you can't process the body streamed to you fast enough, then the TCP window will fill and the server will stop sending. The alternative--empty the buffer in the kernel at the earliest--can lead to memory spikes in your program if you request large bodies in a short amount of time. In my experience, you should aim for sustained memory load rather than spiky memory load if you can.

What likely happens is that the http client ends up in a state where it knows the status code of the request, but it hasn't yet read all of the body. Then the cancel arrives on the context. In this case, we abort the body read. It can be a half read, or perhaps we have not yet pulled anything from the body into the Go process from the kernels buffer.

One reason for this behavior may be that many HTTP requests have no body, so you end up with code which is more complex if you need to cater for this and the context at the same time.

Silviu already wrote what I would do to alleviate the problem.