Blocking waitgroup

[vyasgir...@gmail.com]

I am implementing a multi part file downloader and I need help to find the blocking element in the following piece of code.

func downPart(wg *sync.WaitGroup, url string, dataChan chan []byte, range1, range2 int) {


 defer wg.Done()
 client := new(http.Client)
 req, _ := http.NewRequest("GET", url, nil)


 req.Header.Set("Range", strconv.Itoa(range1)+"-"+strconv.Itoa(range2))
 data, err := client.Do(req)
 if err != nil {
 fmt.Println(err)
 return
 }
 dataByte := new(bytes.Buffer)
 dataByte.ReadFrom(data.Body)
 fmt.Println("Done")


 dataChan <- dataByte.Bytes()
}
func multiDownload(url string, length int) bool {
 var wg sync.WaitGroup


 x := 4
 split := length / x
 fmt.Println(length)
 dataChan := make([]chan []byte, x)
 for i := 0; i < x; i++ {
 wg.Add(1)


 range1 := i * split
 range2 := (i+1)*split - 1
 if range2 == length-2 {
 range2 = length
 }
 fmt.Println(len(dataChan), range1, range2)
 go downPart(&wg, url, dataChan[i], range1, range2)
 }


 var data []byte
 wg.Wait()


 for i := 0; i < x; i++ {
 fmt.Println("waiting")
 data = <-dataChan[i]
 fmt.Println(data)
 }
 return true

}

[Justin Israel]

Hi,

I'm seeing a number of issues with your example code...

You are creating a slice of channels when you really could just use a single channel to receive all of the results.

You wait on a WaitGroup, but the goroutines only call Done() *after* they are able to send their result on the channel. But, you don't start receiving from the channel until after the WaitGroup unblocks. So this deadlocks.

Even after the WaitGroup issue is resolved, the channels in that slice have actually never been created. They are nil channels, and you still end up in a deadlock when trying to receive from them.

I've created a simplified version of your example which can run in the playground: https://play.golang.org/p/V4jy8iD5pr

You can see the two spots where you can comment out the WaitGroup, and make() the channels, which then allows the program to succeed and receive the values from the goroutines. But to be honest, this code can still be simplified further, since you don't need to slice of channels. You already know how many workers you are starting up, so you can either:

... ditch the WaitGroup altogether and just loop on the expected results: https://play.golang.org/p/gtogLx9cJa

... or if you wanted to write the receiving logic in a way where you don't have to care how many results to expect, you can make use of the WaitGroup to control when to close the channel and end the receiving loop: https://play.golang.org/p/c4JrMHSD4x

Justin

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave Cheney]

When your program blocks, hit control-\ and you'll get a stack trace of every running goroutine, the ones blocked inside downPart will point to the place they are blocked.

[Vyas Giridharan]

Thanks.