keeping main() alive while goroutine complete

[kortschak]

Hi,

What is an idiomatic way to keep main() alive while communicating
goroutines complete? In the following gist https://gist.github.com/1075232
I have the relevant sections of a small program that uses a collection
of workers and one output server that writes data to file.
processServer sends results on buffer directly to the output
goroutine.

I have tried using sync.WaitGroup (wg.Add(1) at line 8.5 and wg.Done
in a defer'd func in processServer, and then wg.Wait()'ing before
shutting down the output goroutine, but this lead to a deadlock (I'm
at a loss to understand this), so I've ended up just waiting a small
amount of time at critical points (which seems very kludgy) and
watching the buffer channel's length.

Suggestions about this setup (including approach to communication
between the goroutines) and how to improve it are appreciated.

thanks
Dan

[Vida]

This should do it. Note that this uses the synchronous feature to sync
the main with the goroutine.

package main

import "fmt"

func main() {
c := make(chan bool)
go test(1,c)
go test(2,c)
go test(3,c)
<-c
<-c
<-c

fmt.Println("DONE MAIN")
}


func test(i int, c chan bool) {
fmt.Println(i)
c<-true

}



On Jul 11, 9:43 am, kortschak <dan.kortsc...@adelaide.edu.au> wrote:
> Hi,
>
> What is an idiomatic way to keep main() alive while communicating

> goroutines complete? In the following gisthttps://gist.github.com/1075232

[Vida]

Note that every function will get called but will get stuck at c <-
true until the second part of main <-c gets called 1 by 1.

[Athiwat Chunlakhan]

Oh god, I didn't read the whole thing, sorry.

[Jessta]

On Mon, Jul 11, 2011 at 12:43 PM, kortschak
<dan.ko...@adelaide.edu.au> wrote:
> Hi,
>
> What is an idiomatic way to keep main() alive while communicating
> goroutines complete? In the following gist https://gist.github.com/1075232
> I have the relevant sections of a small program that uses a collection
> of workers and one output server that writes data to file.
> processServer sends results on buffer directly to the output
> goroutine.

You likely want something like this, https://gist.github.com/1075298
When you use WaitGroup you have to be careful not to get in the way of
any channel communication.
You'll get deadlocks if you're waiting for a goroutine to end and that
goroutine is waiting to send on a channel.

> I have tried using sync.WaitGroup (wg.Add(1) at line 8.5 and wg.Done
> in a defer'd func in processServer, and then wg.Wait()'ing before
> shutting down the output goroutine, but this lead to a deadlock (I'm
> at a loss to understand this), so I've ended up just waiting a small
> amount of time at critical points (which seems very kludgy) and
> watching the buffer channel's length.
>
> Suggestions about this setup (including approach to communication
> between the goroutines) and how to improve it are appreciated.
>
> thanks
> Dan

--
=====================
http://jessta.id.au

[kortschak]

Yeah, that's essentially what I have tried, though the wg.Add(1) is
called prior to the go call and the wg.Done() is called prior to
completion of processServer in the body of that func - that is what
gives me the deadlock.

The core of processServer is this

func processServer(index *interval.Tree, queue, output chan
*feat.Feature[, optional wg *sync.WaitGroup here]) {
[define vars]

for feature := range queue {
[do stuff]
output <- feature
}

[optional wg.Done() here]
}

thanks
Dan

On Jul 11, 1:12 pm, Jessta <jes...@jessta.id.au> wrote:
> On Mon, Jul 11, 2011 at 12:43 PM, kortschak
>

> <dan.kortsc...@adelaide.edu.au> wrote:
> > Hi,
>
> > What is an idiomatic way to keep main() alive while communicating

> > goroutines complete? In the following gisthttps://gist.github.com/1075232

> > I have the relevant sections of a small program that uses a collection
> > of workers and one output server that writes data to file.
> > processServer sends results on buffer directly to the output
> > goroutine.
>

> You likely want something like this,https://gist.github.com/1075298

[Kyle Lemons]

The core of processServer is this

func processServer(index *interval.Tree, queue, output chan
*feat.Feature[, optional wg *sync.WaitGroup here]) {
       [define vars]

       for feature := range queue {
               [do stuff]
               output <- feature
       }

       [optional wg.Done() here]
}

Basic rules for using WaitGroup:

1. Call wg.Add(1) before you start the goroutine (go func())

2. Defer thecall to wg.Done() inside the goroutine (defer wg.Done())

3. Don't call wg.Wait() until after the Add/go loop is done

4. If you call Add outside of the goroutine that will call Wait(), only call it before you call wg.Done(), in a goroutine that has been accounted for by an Add, and abide by the same rules above. 

[Dan Kortschak]

I *thought* I had followed all those rules in one of my tries at getting
it to work (after the advice you gave last time). Clearly I didn't as I
am having no problems after working through those formally.

thanks for your time.
Dan