Newbie Concurrency Question, detecting stdin close + waiting for handlers to complete + event loop

[dave josephsen]

Hey guys,

I'm very new to Go, and am having a great time, and would like your opinion on the following pretty simple concurrency problem.  It's a stdin input handler toy where I have one goroutine listening to stdin, and another handling the input, while main uses a select to arbitrate between them:

package main

import (
   "fmt"
   "bufio"
   "os"
)

func getInput(inChan chan string){
   input := bufio.NewScanner(os.Stdin)
   for input.Scan(){
      inStr:=input.Text()
      inChan <- inStr
   }
   haveInput <- false
}

func handleInput(inStr *string, outChan chan string){
   outChan <- fmt.Sprint("Handled: ",*inStr)
}

func main(){
   inChan, outChan := make(chan string), make(chan string)

   go getInput(inChan)
   for { //event loop start
      select {
         case inStr:= <-inChan:
            go handleInput(&inStr,outChan)
         case outStr:= <-outChan:
            fmt.Println(outStr)
      }
   }
}

At this point I've read about the various channel patterns in the docs, but am having a little trouble applying them here (I'm sure it's just me).  This program works fine if you call it like:  

# program

and then feed it input.  But when you call it like:

# echo foo | program

It either panics or hangs when stdin goes away (because getInput() returns and handleInput() sleeps and we hang in the select forever).  So I'm wondering what sort of messaging pattern I should employ given that I want to:

1. detect that stdin went away but
2. don't exit main() until all handleInput()'s are finished.
3. keep looping as long as stdin is open

Various answers come to mind:

 A mutexed int for handleInput() coupled with a global bool that goes true when stdin is dead... but this doesn't feel like "communicating to synchronize"

An stdinDead Channel coupled with a noMoreHandlers  Channel .... but this feels like a preponderance of channels

Other combinations of those... but given the elegance of the patterns in the docs, I feel like I'm missing something simple, and welcome your advice.

Thanks

-dave

[Dave Cheney]

Your sample program did not compile. I've made some changes, please
try this version which uses a channel to signal to your event loop
that there is not more input.

http://play.golang.org/p/XiifMw_8ZB

[dave josephsen]

Thanks Dave!

Apologies for the broken code. I pasted from the wrong term.

So yes, I understand the signaling technique you've employed, but my original problem persists.  

So if, for example, I added a time.Sleep to the handler function like so: http://play.golang.org/p/ZzaB8y53DH

Then sent ten or so lines to it via the shell like so:

echo 'line 1
line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9
line 10' | myprogram

I won't get any output because the inputHandler()'s will not have had a chance to finish before getInput() signals done. 

So you see I'm looking for a way to signal both that there's no input (so the program doesn't deadlock on IO) and also that the handlers are all done (so we don't exit too early). Again I have a few notions of how I could do this, but they all seem... un-go-like to me given most the examples I've read so far.

[Dave Cheney]

As a suggestion, stop using channels, all the actions of this program are synchronous as none of your channels are buffered. 

At least stop processing the input on another channel. 

I haven't really though about it more than that. 

Cheers

Dave

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[dave josephsen]

Yes, as written, it's not a compelling use-case for concurrency, I agree.

An interesting feature of select with channels in Go is that I can pretty much epoll anything that I wrap in a goroutine (as opposed to only being able to epoll things that have an FD interface).  So my goal wasn't so much to "be parallel" as much as prove out the notion that I might have some input source, and some handlers, and by wrapping them both in goroutines, can select between them in an add-hoc way, like I would with an event-processing library.

The examples I see that involve signal-passing with channels in Go seem to focus on what I might call "single state" signaling; we've launched some gaggle of goroutines that are all operating independently on the same task, and we need to signal that they're all done before we can move on.   So I'm just wondering if there's a commonly-used syntax in the community that elegantly expresses that we have a few input threads out over there, and some different types of handlers over here, and we want to loop until there's no more input and all the handlers are finished executing, which is kind of a 'multi state' signaling pattern.