os/exec stdout bytes.Buffer and timeout behavior?

[Alex Buchanan]

package main

import (

"bytes"

  "context"

  "log"

  "os/exec"

  "time"

)

func main() {
    var stdout bytes.Buffer

  ctx, cancel := context.WithTimeout(context.Background(), time.Second)

  defer cancel()

  cmd := exec.CommandContext(ctx, "/bin/sh", "-c", "sleep 10; echo foo")

  cmd.Stdout = &stdout

  err := cmd.Run()

  log.Printf("%s, %s, '%s'", err, ctx.Err(), stdout.String())

}

This runs for 10 seconds, but I was expecting 1 second. Can someone help me understand what is happening? I think it has something to do with the stdout bytes.Buffer?

[Ian Lance Taylor]

Yes. The problem is that when the context passed to CommandContext
expires, it kills the process started by the command, but does not
kill any subprocesses that that command may have started. So when the
context expires the /bin/sh is killed, but the sleep subprocess is
still running. Because you use a bytes.Buffer, the program has
created a pipe to capture the standard output of the command. After
the process dies, cmd.Run is waiting for that pipe to be closed to
make sure that it gathers all the data. But the pipe is held open by
the sleep subprocess. It is only after that subprocess exits that the
pipe is closed and your program continues.

Ian

[Alex Buchanan]

Ok, that's a great explanation, thanks. Do you have a recommendation for how to exit early? Would StdoutPipe() help? I tried setting Setpgid to true, but probably os/exec doesn't pay attention to process groups.

Alternatively, I can configure my shell scripts to close stdout/err on exit? Not pretty, but might work.

My best approach so far is to use time.After and panic, instead of context. This code is being used in a test, so panic is somewhat acceptable.

Thanks again.

[Ian Lance Taylor]

On Fri, Sep 29, 2017 at 1:33 PM, Alex Buchanan <buch...@ohsu.edu> wrote:
>
> Ok, that's a great explanation, thanks. Do you have a recommendation for how to exit early? Would StdoutPipe() help? I tried setting Setpgid to true, but probably os/exec doesn't pay attention to process groups.
>
> Alternatively, I can configure my shell scripts to close stdout/err on exit? Not pretty, but might work.
>
> My best approach so far is to use time.After and panic, instead of context. This code is being used in a test, so panic is somewhat acceptable.

I would say that CommandContext is really only useful for simple
programs, not for shell scripts. So using time.After seems reasonable
to me.

Ian

[roger peppe]

One might argue that Run should respect the context deadline even in the presence of pipes that don't EOF.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ian Lance Taylor]

On Sun, Oct 1, 2017 at 2:40 PM, roger peppe <rogp...@gmail.com> wrote:
> One might argue that Run should respect the context deadline even in the
> presence of pipes that don't EOF.

Fair point.

Ian

>> email to golang-nuts...@googlegroups.com.

[an...@aubble.com]

There is an open issue regarding this behaviour. https://github.com/golang/go/issues/21922