Length of Channel Using len Function

[dc0d]

TL;DR

Does assigning a (buffered) channel, already in a variable, to a second variable, affects the result of len function?

Long version:

What is happening here? - Code at the end; Go 1.7.

Output 1:

Nine times:

[  info ] 2016/08/28 19:51:28 LEN_BEFORE=0                                    
[  info ] 2016/08/28 19:51:28 LEN=7                                          
[  info ] 2016/08/28 19:51:28 S00=7

But if the second case gets commented like this:

case <-limiter.C:
 // if len(actualBuffer) > 0 {
 // buffer = actualBuffer
 // } else {
 // buffer = nil
 // }

It works as expected; Output2:

[  info ] 2016/08/28 19:54:28 LEN_BEFORE=0                                    
[  info ] 2016/08/28 19:54:28 LEN=7                                          
[  info ] 2016/08/28 19:54:28 S00=7                                          
[  info ] 2016/08/28 19:54:29 LEN_BEFORE=7                                    
[  info ] 2016/08/28 19:54:30 LEN_BEFORE=7                                    
[  info ] 2016/08/28 19:54:31 LEN_BEFORE=7                                    
[  info ] 2016/08/28 19:54:32 LEN_BEFORE=7                                    
[  info ] 2016/08/28 19:54:33 LEN_BEFORE=7                                    
[  info ] 2016/08/28 19:54:34 LEN_BEFORE=7                                    
[  info ] 2016/08/28 19:54:35 LEN_BEFORE=7                                    
[  info ] 2016/08/28 19:54:36 LEN_BEFORE=7                
...

Code:

package main


import (
 "log"
 "time"


 "github.com/comail/colog"
)


func status00Channeler() {
 <-start


 limiter := time.NewTicker(time.Second / maxMsgPerSec)
 fetchLimiter := time.NewTicker(time.Second)


 var buffer chan *Data
 actualBuffer := make(chan *Data, maxMsgPerSec)
 db, err := newDB()
 if err != nil {
 log.Panic(err)
 }


FIRST:
 for {
 select {
 case <-interrupted:
 break FIRST
 case <-limiter.C:
 // if len(actualBuffer) > 0 {
 // buffer = actualBuffer
 // } else {
 // buffer = nil
 // }
 case i := <-buffer:
 select {
 case status00 <- i: // will block here
 case <-interrupted:
 break FIRST
 }
 case <-fetchLimiter.C:
 log.Printf("LEN_BEFORE=%d", len(actualBuffer))
 if len(actualBuffer) > 0 {
 continue
 }


 s00, err := db.GetIncomings()
 if err != nil {
 log.Println(`error:`, err)
 time.Sleep(time.Second)
 continue
 }
 if s00 == nil || len(s00) == 0 {
 continue
 }


 FILL_BUFFER:
 for _, v := range s00 {
 select {
 case actualBuffer <- v:
 default:
 break FILL_BUFFER
 }
 }


 log.Printf("LEN=%d", len(actualBuffer))
 log.Printf("S00=%d", len(s00))
 }
 }
}


func main() {
 go status00Channeler()
 close(start)


 <-time.After(time.Second * 30)
}


var (
 status00 = make(chan *Data, maxMsgPerSec)
)


type Data struct{}


const (
 maxMsgPerSec = 60
)


func newDB() (*DB, error) {
 res := new(DB)
 return res, nil
}


func (db *DB) GetIncomings() ([]*Data, error) {
 var res []*Data
 res = append(res, &Data{})
 res = append(res, &Data{})
 res = append(res, &Data{})
 res = append(res, &Data{})
 res = append(res, &Data{})
 res = append(res, &Data{})
 res = append(res, &Data{})


 return res, nil
}


type DB struct{}


func init() {
 colog.Register()
}


var (
 start       = make(chan struct{})
 interrupted = make(chan struct{}) // comes from sys interrupts SIGINT, SIGTERM, etc
)

[Jan Mercl]

Using len(ch) like this in a concurrency scenario is a big no because then the value you get carries 0 bits of useful information. It's not a data race, it's worse, the race is semantic and not fixable without removing the use of len(ch).

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

-j

[dc0d]

Would you please elaborate on that?

As I understand it there is no concurrent use of len happening here. It's a for loop and all calling to len is happening sequentially. Unless the channels make the code inside cases of one select statement concurrent - which will be super confusing for me.

[Dave Cheney]

TLDR; channels, like maps, are references to the a data structure stored elsewhere. Calling Len on either returns the length of the data stored in the underlying channel. This is different to a slice, which is a three word value, containing it's own Len and cap values.

As Jan notes, Len(ch) is usually not what you want becuase any value returned from that expression is considered stale in the presence of multiple goroutines.

[Henrik Johansson]

The race is between the len call and the use of the channel. Entries can have been added in between.

[dc0d]

Thanks for comments. The status00 should be not-buffered (status00 = make(chan *Data)); was rate limiting on two channels.

[Klaus Post]

On Sunday, 28 August 2016 17:33:10 UTC+2, Jan Mercl wrote:

Using len(ch) like this in a concurrency scenario is a big no because then the value you get carries 0 bits of useful information. It's not a data race, it's worse, the race is semantic and not fixable without removing the use of len(ch).

In general I would agree, if you know other things, it *can* contain useful information.

I recently had some code, where using "len(channel)" was actually useful, since it allowed me to do a proper shutdown of a queue+retry queue with relatively simple code. I tried to boil it down as an example: 

https://play.golang.org/p/DMcl8k5Ssx

I want to receive messages to a bunch of workers, but they must be able to add the entry to a "retry queue" under certain conditions. If I just add them back to the original channel I risk a deadlock, so I have i prioritized queue (channel) for that. Furthermore I must ensure that all messages are processed when we exit, that means both the queue and retry queue must be empty.

In this case, using the length is useful, since it ensure that the last worker will not exit if there are tasks still in the retry queue - since only tasks (and functions called by the task) can add stuff to the retry queue.

For this, I found len(channel) to be both useful and aiding (for what I can come up with) the simplest solution. I think that in my 4 years of programming Go, it is probably the first time I have encountered it ;)

/Klaus