Breaking an io.Pipe loop

[Salvatore Domenick Desiano]

I've got some code that is basically Expect-lite written in Go. I've got a deadlock and I'm hoping someone here can help me untangle it. I posted this yesterday but at the time I thought the Expect part of the code wasn't relevant and I left it out. I now believe that part of the code is the problem.

I have:

1. exec.Command() with stdin and stdout replaced with io.Pipe()
2. a Go routine listening to stdout, watching for strings, and writing to stdin

I know that Wait() won't return until stdin is closed. The problem is that I can't tell when I need to close stdin. Since Wait() doesn't return, the main thread can't do it. For some reason stdout doesn't get closed even with the program exits (stdout.Read() is part of the deadlock) so the Expect Go routine can't do it.

The only thing I can think of is another Go routine monitoring cmd.ProcessState but that doesn't feel right.

What am I missing?

Thank you!

-- Salvatore

smile.

[Robert Engels]

Code?

On Jun 17, 2024, at 9:46 PM, Salvatore Domenick Desiano <near...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEQs7S8M_KTESzGPfaQsEHi9gVc1O_M5DAS4nKJ0SDn_Mqm3Tg%40mail.gmail.com.

[Salvatore Domenick Desiano]

Here's a hopefully semantically identical minimal example:

func main() {

cmdName := "/bin/bash"

cmdArgs := []string{"-c", "sleep 5"}

cmdStdinR, cmdStdinW := io.Pipe()

cmdStdoutR, cmdStdoutW := io.Pipe()

go func() {

defer cmdStdinW.Close()

io.Copy(cmdStdinW, cmdStdoutR)

}()

cmd := exec.Command(cmdName, cmdArgs...)

cmd.Stdout = cmdStdoutW

cmd.Stdin = cmdStdinR

if err := cmd.Run(); err != nil {

fmt.Println(err)

}

}

This program deadlocks after seconds. The three locked Go routines are the io.Copy( (stuck inside io.Copy because cmdStdoutR doesn't close), the exec.Command Go routine (also stuck in an io.Copy) and the main routine (stuck in cmd.Wait()).

[Salvatore Domenick Desiano]

A bunch of typos. The last paragraph should have been:

This program deadlocks after 5 seconds. The three deadlocked Go routines are the explicit go func() (stuck on io.Copy() because cmdStdoutR never closes), exec.Command's internal Go routine copying data to stdin also (stuck on io.Copy() because cmdStdinR never closes), and the main routine (stuck on cmd.Wait()).

[Robert Engels]

Nothing is writing and closing StdInR so it will never complete. 

On Jun 17, 2024, at 9:54 PM, Salvatore Domenick Desiano <near...@gmail.com> wrote:



A bunch of typos. The last paragraph should have been:

This program deadlocks after 5 seconds. The three deadlocked Go routines are the explicit go func() (stuck on io.Copy() because cmdStdoutR never closes), exec.Command's internal Go routine copying data to stdin also (stuck on io.Copy() because cmdStdinR never closes), and the main routine (stuck on cmd.Wait()).

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEQs7S9e8bmCd7cFAzDTu5g3rUQnuT7iaKo%2BLRR-J1nH%3D9%2BNBw%40mail.gmail.com.

[Salvatore Domenick Desiano]

On Mon, Jun 17, 2024 at 11:39 PM Robert Engels <ren...@ix.netcom.com> wrote:

Nothing is writing and closing StdInR so it will never complete. 

I may have this wrong, but isn't io.Copy writing to cmdStdinW and therefore to cmdStdinR?

As for the closing... that's the point. I only want to close cmdStdinR in response to something from cmdStdoutW, whether it is because the command was killed, it exited cleanly, or it output something I'm looking for.

The question is: how can I get notified that the command exited so that I can close its stdin?

[Salvatore Domenick Desiano]

Or maybe put another way:

Imagine I wanted to write a Go program that ran an external executable using exec.Command and piped the executable's stdout back to its stdin until the executable exited of its own accord. How can that be done without a deadlock?

[Ian Lance Taylor]

On Mon, Jun 17, 2024 at 7:49 PM Salvatore Domenick Desiano
<near...@gmail.com> wrote:
>
> Here's a hopefully semantically identical minimal example:
>
> func main() {
>
> cmdName := "/bin/bash"
> cmdArgs := []string{"-c", "sleep 5"}
> cmdStdinR, cmdStdinW := io.Pipe()
> cmdStdoutR, cmdStdoutW := io.Pipe()
> go func() {
>
> defer cmdStdinW.Close()
>
> io.Copy(cmdStdinW, cmdStdoutR)
>
> }()
> cmd := exec.Command(cmdName, cmdArgs...)
> cmd.Stdout = cmdStdoutW
> cmd.Stdin = cmdStdinR
> if err := cmd.Run(); err != nil {
>
> fmt.Println(err)
>
> }
>
> }
>
> This program deadlocks after seconds. The three locked Go routines are the io.Copy( (stuck inside io.Copy because cmdStdoutR doesn't close), the exec.Command Go routine (also stuck in an io.Copy) and the main routine (stuck in cmd.Wait()).

I haven't checked whether this is the problem, but as someone else
pointed out this is a strange place to use io.Pipe. Much better to
use os.Pipe. Much better still to use exec.Cmd.StdinPipe and
exec.Cmd.StdoutPipe.

Ian

[Harri L]

I pointed out that maybe exec.Cmd.StdinPipe and exec.Cmd.StdoutPipe would be better. Here’s the modified sample from this thread and the complete program for easier copying to play with—no deadlock with this.

func main() { defer err2.Catch() cmdName := "/bin/bash" cmdArgs := []string{"-c", "sleep 5"} cmd := exec.Command(cmdName, cmdArgs...) cmdStdinW := try.To1(cmd.StdinPipe()) cmdStdoutR := try.To1(cmd.StdoutPipe()) go func() { defer err2.Catch() // No need to close the pipes try.To1(io.Copy(cmdStdinW, cmdStdoutR)) fmt.Println("copier ends") }() try.To(cmd.Start()) // we cannot use just Run() with Cmd.XxxPipe calls... try.To(cmd.Wait()) // ... according to docs fmt.Println("all OK") }

The entire program sample:

package main import ( "fmt" "io" "os/exec" "github.com/lainio/err2" "github.com/lainio/err2/try" ) func main() { defer err2.Catch() cmdName := "/bin/bash" cmdArgs := []string{"-c", "sleep 5"} cmd := exec.Command(cmdName, cmdArgs...) cmdStdinW := try.To1(cmd.StdinPipe()) cmdStdoutR := try.To1(cmd.StdoutPipe()) go func() { defer err2.Catch() // No need to close the pipes try.To1(io.Copy(cmdStdinW, cmdStdoutR)) fmt.Println("copier ends") }() try.To(cmd.Start()) // we cannot use just Run() with Cmd.XxxPipe calls... try.To(cmd.Wait()) // ... according to docs fmt.Println("all OK") }

​