Protective buffered channel that never triggers deadlock

[Zhaoxun Yan]

Since I found if inserting into a buffered channel could cause a crash if it is full already

( now I only use unbuffered channels in work)

https://groups.google.com/g/golang-nuts/c/U8lz6noKkuA

package main

var c = make(chan int, 1)

func main() {
   
    c <- 1
    c <- 2 //fatal error: all goroutines are asleep - deadlock!
    c <- 3

}

I made up a work around that just dumps the new message if the buffer is full:

package main

import (
    "fmt"
    "sync"
    )

type ChanInt struct{
    Ch chan int
    Mu *sync.Mutex
}

var c = ChanInt{make(chan int, 1), new(sync.Mutex)}

func (ch ChanInt) Fill(k int){
    ch.Mu.Lock()
    defer ch.Mu.Unlock()
   
    if len(ch.Ch) < cap(ch.Ch){
        ch.Ch <- k
    }
}

func main() {
   
    fmt.Println("Channel Status: ",len(c.Ch), cap(c.Ch))
    c.Fill(5)
    fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
    fmt.Println("get one out of Channel:", <-c.Ch)
    fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
   
    c.Fill(6)
    // c.Ch <- 7 //fatal error: all goroutines are asleep - deadlock!
    c.Fill(7)
    fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
    fmt.Println("get one out of Channel:", <-c.Ch)
}

[Shulhan]

I believe the idiom way is to use select to check if channel is full or
not.

See "leaky buffer" section on https://go.dev/doc/effective_go#channels .

--
{ "github":"github.com/shuLhan", "site":"kilabit.info" }

[Robert Engels]

Adding to a full buffered channel will not cause a crash. It can deadlock just as the unbuffered can - but the deadlocks can be harder to diagnose. 

On Apr 13, 2022, at 3:40 AM, Zhaoxun Yan <yan.z...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9ca5c13b-7797-4d64-bc2c-1f1ff7281168n%40googlegroups.com.

[Brian Candler]

> I made up a work around that just dumps the new message if the buffer is full:

You don't need to do that: use select { ... } to check if the channel is ready to receive or not.

https://go.dev/play/p/Mb7VVCTvnfk

But in practice, this is almost never needed, because one goroutine is doing the sending and a different goroutine is doing the receiving.  The sending goroutine blocks until the receiver is ready.

[Robert Engels]

That is not a general idiom. That is a specialized case that maps the buffer size on a channel to the size of an ancillary structure.

In general if you are spinning waiting for channels you’ve done something wrong. Go has no concept of cpu affinity so the practice has little value. 

(Note that the leaky buffer example does not spin / busy wait)

On Apr 13, 2022, at 4:44 AM, Robert Engels <ren...@ix.netcom.com> wrote:



[Zhaoxun Yan]

You are right, Brian.

The difference is not whether the channel is buffered or unbuffered - while the receiver is present, none will encounter the deadlock.

package main

import (
    "fmt"
    "time"
    )

//var c = make(chan int)

var c = make(chan int, 2)

func report(){
    for{
        select{
        case n:= <- c:
            fmt.Println(n, time.Now().Format("2006-01-02 15:04:05"))
        default:
            time.Sleep(time.Second)
        }
    }
}

func main() {
   
    go report()
   
    for i:=1; i<=5; i++{
        c <- i
    }

}

But obviously, the buffered channel provides another layer of support in case the receiver is temporarily absent:

package main

import (
    "fmt"

    "time"
    )

//var c = make(chan int)
var c = make(chan int, 5)

func report(){
    for{
        select{
        case n:= <- c:
            fmt.Println(n, time.Now().Format("2006-01-02 15:04:05"))
        default:
            time.Sleep(time.Second)
        }
    }
}

func main() {
   
    for i:=1; i<=5; i++{
        c <- i
    }
   
    report()

}

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/6ExktXrF5Xc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/7df07f32-d7aa-45db-91d6-de75e18ad8d8n%40googlegroups.com.

[Brian Candler]

The fact that the sender may have to wait does not by itself make a "deadlock".

It's a deadlock if the sender blocks and, due to poor programming, the receiver is *never* ready to receive. That's a problem with your software, not with Go's concept of channels.

Blocking on send until the channel is ready is exactly what is required normally. Are you suggesting something else should happen? If so, what exactly?

This is well worth watching:

Rethinking Classical Concurrency Patterns (Bryan C. Mills)
https://www.youtube.com/watch?v=5zXAHh5tJqQ

[Zhaoxun Yan]

Something bizarre happened in CGO. My new code started the anonymous goroutine to receive first but it does not sleep below the goroutine and jumps to end the function. So I  returned to the sending before receiving way, which guaranteed the program's safety.

There ain't always one way that fits all.  Sometimes a public variable is better than a channel.

Knowing the boundaries of your tool is always better than believing it will always prevail.

Blocking on send until the channel is ready is exactly what is required normally.

 " Are you suggesting something else should happen? If so, what exactly?"

Beyond the cgo accident, Here is one scenario:

When one server finds something wrong, it sends out a request to another server for help.

Then it makes a log and then a ticker to recheck the situation again and again and in the meantime, to receive a response from that server that indicates everything is fine.

So here is a gap between sending help and getting ready to receive the "ok" response. It is a tiny gap but it is not possible that the other server sends the "ok" message back through TCP connection just in this gap. Yes my program handles the TCP message as always, in this case sends back through the channel to the goroutine that has not prepared the select for ticker channel and  ok-receiving channel. Uh-oh, deadlock, crash! What is your suggestion? buffered channel? In this case you already admit that opportunity that the sending happens when the receiver is not ready.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0c91adbb-94c7-4475-891d-3c4bb3d4e80dn%40googlegroups.com.

[Zhaoxun Yan]

This code really blows my mind, Brian! I didn't know select can be flexible as this:

package main

import (
    "fmt"

)

var Ch = make(chan int, 1)

func Fill(ch chan int, k int) {
    select {
    case ch <- k:
        // sent
    default:
        // discard
    }
}

func main() {

    fmt.Println("Channel Status: ", len(Ch), cap(Ch))
    Fill(Ch, 5)
    fmt.Println("Channel Status: ", len(Ch), cap(Ch))
    fmt.Println("get one out of Channel:", <-Ch)
    fmt.Println("Channel Status: ", len(Ch), cap(Ch))

    Fill(Ch, 6)
    // Ch <- 7 //fatal error: all goroutines are asleep - deadlock!
    Fill(Ch, 7)
    fmt.Println("Channel Status: ", len(Ch), cap(Ch))
    fmt.Println("get one out of Channel:", <-Ch)
}

[Brian Candler]

On Friday, 15 April 2022 at 06:47:09 UTC+1 yan.z...@gmail.com wrote:

When one server finds something wrong, it sends out a request to another server for help.

Then it makes a log and then a ticker to recheck the situation again and again and in the meantime, to receive a response from that server that indicates everything is fine.

So here is a gap between sending help and getting ready to receive the "ok" response. It is a tiny gap but it is not possible that the other server sends the "ok" message back through TCP connection just in this gap. Yes my program handles the TCP message as always, in this case sends back through the channel to the goroutine that has not prepared the select for ticker channel and  ok-receiving channel. Uh-oh, deadlock, crash! What is your suggestion? buffered channel?

TCP sockets and Go channels are two completely different things.

But in either case, if a receiver becomes ready before the sender, it's fine.  The receiver just waits for data to arrive.

And if the receiver is not ready first, it's also fine.  The sender will send up to the buffer size (in the case of a buffered channel), and after that it will pause until the receiver is ready.  (In the case of TCP you have quite a lot of buffering in network sockets; typically it could send a megabyte or more before it pauses).

A pause is not a "deadlock".  A pause is just waiting until it's OK to proceed.

In this case you already admit that opportunity that the sending happens when the receiver is not ready.

And there is no problem at all. It really doesn't matter if the sender becomes ready first, or the receiver becomes ready first.  Here are two concrete examples, using an unbuffered channel:

https://go.dev/play/p/r5eptx4dE7T

https://go.dev/play/p/sTB5lio6Rhx

Aside: in those examples I have used a sync.WaitGroup to ensure that the main program doesn't terminate until both the goroutines have completed.

[Zhaoxun Yan]

Thanks for your demonstration, Brian.

Actually my code involving cgo is quite like your first case, like this code:

log("start testing")

go func{

  for

      select

      case a: <=receiving

      case <- killsig

...

}()

subscribequotations

( meanwhile the cgo of the dll will call back go functions to fill receiving channel)

time.Sleep(3 * time.Seconds)

disconnect and log

killsig <- true

Strangely I found the start testing and disconnect log clustered and the disconnect  did actually happen. Then I switch back to the sequential case that the receiving channel gots filled without receiving until disconnection. It works now.

I am really curious about your example on select -  is it a try... except alternative? Can it be used in other scenarios?

func Fill(ch chan int, k int) {
    select {
    case ch <- k:
        // sent
    default:
        // discard
    }
}

--

You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/6ExktXrF5Xc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9a0a275e-54cf-40da-b0ac-8ce724fc3d49n%40googlegroups.com.

[Brian Candler]

On Friday, 15 April 2022 at 08:31:17 UTC+1 yan.z...@gmail.com wrote:

Thanks for your demonstration, Brian.

Actually my code involving cgo is quite like your first case, like this code:

log("start testing")

go func{

  for

      select

      case a: <=receiving

      case <- killsig

...

}()

subscribequotations

( meanwhile the cgo of the dll will call back go functions to fill receiving channel)

time.Sleep(3 * time.Seconds)

disconnect and log

killsig <- true

That code is obviously incomplete, but I'm guessing when you receive a value on killsig you return from the function.

A better Go idiom for that is to close a channel.  All the receivers on that channel will see that the channel is closed:

    case msg, ok <- killsig:

        // msg is the zero value and ok is false if the channel is closed

The same channel can be shared between many receivers, so it becomes a way of broadcasting a kill message.  See https://go.dev/tour/concurrency/4

You should never *send* on a closed channel though - that will cause a panic.

 

I am really curious about your example on select -  is it a try... except alternative?

It's not really anything like "try... except".

The "select" statement is specifically for channel communication.  See the description in the Go Tour starting here:

https://go.dev/tour/concurrency/5  # pages 5 and 6

It picks one branch where a channel is ready to send or receive, or the "default" branch if none are.

If you have a series of general conditions that you want to try one by one, then there is the "switch" statement:

https://go.dev/tour/flowcontrol/9   # pages 9 to 11

In most languages "try... except" is an error recovery mechanism which can handle unwinding the stack (returning from multiple levels of nested function calls).  The nearest equivalent to that in go is panic/recover, but that should only be used in very special circumstances.

[Zhaoxun Yan]

" Strangely I found the start testing and disconnect log clustered and the disconnect  did actually happen. Then I switch back to the sequential case that the receiving channel gots filled without receiving until disconnection. It works now."

-- I found the error occurred again. It turned out that it referenced a global integer variable which might be 0 yet so it called the disconnection function even before it got connected.  It has nothing to do with first to receive or to send.

--

You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/6ExktXrF5Xc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/452795b8-4817-4161-84e7-02665e24ae06n%40googlegroups.com.