Closing write side of os.Pipe caused different behaviors

[huan xiong]

Hi, I wonder why the two programs below have different behaviors? Both start `yes` command, close write side of the pipe (because the main process doesn't use it), and read from the read side of the pipe. The only difference is one doesn't use goroutine and another uses it. 

1) The first program doesn't use goroutine. It works as expected (you'll need to press Ctl-C to temrinate the program).

```
package main

import (
    "io"
    "os"
    "os/exec"
)

func main() {
    rpipe, wpipe, err := os.Pipe()
    if err != nil {
        panic(err)
    }

    cmd := exec.Command("yes")
    cmd.Stdout = wpipe
    if err := cmd.Start(); err != nil {
        panic(err)
    }

    wpipe.Close()

    if _, err := io.Copy(os.Stdout, rpipe); err != nil {
        panic(err)
    }
}
```

2) The second program is almost same as the first one, except that it wraps the `yes` command related code in a goroutine. It fails to read data from pipe. The culprit is `wpipe.Close()` line.

```
package main

import (
    "io"
    "os"
    "os/exec"
    "fmt"
)

func main() {
    rpipe, wpipe, err := os.Pipe()
    if err != nil {
        panic(err)
    }

    go func() {
        cmd := exec.Command("yes")
        cmd.Stdout = wpipe
        if err := cmd.Start(); err != nil {
            panic(err)
        }
    }()

    // Commenting out this line would make the program works fine. Why?
    wpipe.Close()

    n, err := io.Copy(os.Stdout, rpipe)
    if err != nil {
        panic(err)
    }
    fmt.Printf("n: %d", n)
}

// Output:
// n: 0
```

Question 1: I wonder why the above code doesn't work?

I find two ways to make it work:

1) Remove `wpipe.Close()` line. I don't like this approach because I think it's a convention in Unix programs to close unused file descriptors.

2) Move `wpipe.Close()` to the goroutine. Question 2: Why does this work?

Thanks for any explanation.

[Kurtis Rader]

You have at least three problems. The first is a race condition in the second, goroutine, version of your program. Add a `time.Sleep(time.Second)` just before the `wpipe.Close()` and you should see the same behavior as the non-goroutine version. That provides a hint regarding the primary problem.

The second problem is you should not be closing the write side of the pipe until the process writing to the pipe has terminated.

The third, and primary, problem is a misunderstanding of what it means to assign a `os.File` object (such as from `os.Pipe`) to the `cmd.Stdout` structure member and how that interacts with closing that file object. Calling `cmd.Start` creates a second reference to the write side of the pipe. If the write side of the pipe is open when `cmd.Start` is executed calling `wpipe.Close` won't actually close the write side of the pipe. It only removes the first reference to the write side of the pipe. The spawned external process still holds a reference to the write side of the pipe. The reason the second, goroutine, version of your program normally fails is because your program will normally execute the `wpipe.Close` before the goroutine spawns the external process and thus creates a second reference to the write side of the pipe. Which is why adding a tiny sleep before the close "fixes" the problem.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/e175827f-0159-4a2d-8d59-1cad48661002n%40googlegroups.com.

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[huan xiong]

Hi Kurtis,

Thanks for your detailed explanation. It all makes sense to me (I figured out the race condition issue after I posted the question. As it was the first time I posted to the group, I couldn't reply to my own question until the moderator approved it).

For future me and other beginners who run into similar issues, the issue in the second program is that, while the goroutine starting `yes` command appears before `wpipe.Close()` in the code, it isn't necessarily run before that line of the code (I believe this is a golang scheduler implementation detail). So the reliable approach is to move `wpipe.Close()` to the goroutine (It's OK to call it immediately after `cmd.Start()` because command runs in a separate process).

Thanks!