Multi channel design approach

[Mark Laczynski]

I have a scenario which I want to get some feedback on.

Attached is a drawing modeling my situation. There are 3 go routines representing 3 layers/packages in my application. 

1) There's in trigger in Go 1 that initiates a request for some data that is managed by Go 2. It (Go 1) sends a request with a key on reqData chan. 

2) Go 2 returns a pointer to the data on the data chan. 

3) Go 1 does some processing/updates to the data based on the trigger mentioned earlier, 

4) When complete, Go 1 sends a notification to Go 2 saying it's done processing the data, 

5) Go 2 does some of it's own processing on the data relevant to it's own layer.

6) Finally as one of those processing steps, Go 2 notifies Go 3 there was an update to the data by sending it (the updated data) on a chan to Go 3.

My main concern is around the 3 channels I have between Go 1 and Go 2, because I cannot find anyone use this 3 chan approach, and I'm wondering if I'm making some larger mistake that could be solved a better way.

Cheers,

Mark

[Giulio Iotti]

On Sunday, May 31, 2015 at 8:31:40 AM UTC+3, Mark Laczynski wrote:

My main concern is around the 3 channels I have between Go 1 and Go 2, because I cannot find anyone use this 3 chan approach, and I'm wondering if I'm making some larger mistake that could be solved a better way.

I am sorry, but it is a bit too abstract to understand.

After goroutine1 has gotten data from goroutine2, goroutine2 is idle until "done"? Then why not merge the two rooutines into one?

-- 

Giulio

[Egon]

On Sunday, 31 May 2015 08:31:40 UTC+3, Mark Laczynski wrote:

I have a scenario which I want to get some feedback on.

Attached is a drawing modeling my situation. There are 3 go routines representing 3 layers/packages in my application. 

The proper solution highly depends on what the Go1..Go3 are. so what are they? What are you implementing?

 

1) There's in trigger in Go 1 that initiates a request for some data that is managed by Go 2. It (Go 1) sends a request with a key on reqData chan. 

2) Go 2 returns a pointer to the data on the data chan. 

3) Go 1 does some processing/updates to the data based on the trigger mentioned earlier, 

4) When complete, Go 1 sends a notification to Go 2 saying it's done processing the data, 

5) Go 2 does some of it's own processing on the data relevant to it's own layer.

6) Finally as one of those processing steps, Go 2 notifies Go 3 there was an update to the data by sending it (the updated data) on a chan to Go 3.

I would write the interaction piece... e.g.

func Interact(A A, B B, C C) error {

data, err := B.Request(A.GetParams())

if err != nil {

return err

}

err := A.HandleData(data)

if err != nil {

return err

}

B.DataHandled(data)

C.Notify(data)

}

And follow design of it from there...

By breaking up the interaction into different goroutines/objects/types you are making the interaction harder to understand.

 

My main concern is around the 3 channels I have between Go 1 and Go 2, because I cannot find anyone use this 3 chan approach, and I'm wondering if I'm making some larger mistake that could be solved a better way.

You might be overusing channels.

 

Cheers,

Mark

[Mark Laczynski]

I'll make this a bit more concrete.

On Sunday, May 31, 2015 at 2:49:39 AM UTC-4, Giulio Iotti wrote:

On Sunday, May 31, 2015 at 8:31:40 AM UTC+3, Mark Laczynski wrote:

My main concern is around the 3 channels I have between Go 1 and Go 2, because I cannot find anyone use this 3 chan approach, and I'm wondering if I'm making some larger mistake that could be solved a better way.

I am sorry, but it is a bit too abstract to understand.

Go Routine 1 parses a continuous stream. That is its whole responsibility, because there's just a lot of logic in parsing the stream "packets."

Go Routine 2 manages and owns the data. So basically this routine is serializing the access to the data, because various parts of the application can be updating/reading from the data.

Go Routine 3 is an httpHandler that updates the UI. So Go Routine 2 notifies the UI to update the page with the new data.

 

After goroutine1 has gotten data from goroutine2, goroutine2 is idle until "done"? Then why not merge the two rooutines into one?

GoRoutine 2 is blocking after go routine 1 is processing the data. 

The reason I don't have them merged is due to separation of responsibilities. 1 parses the stream, the other controls access to the data.

 

-- 

Giulio

[Mark Laczynski]

On Sunday, May 31, 2015 at 4:36:01 AM UTC-4, Egon wrote:

On Sunday, 31 May 2015 08:31:40 UTC+3, Mark Laczynski wrote:

I have a scenario which I want to get some feedback on.

Attached is a drawing modeling my situation. There are 3 go routines representing 3 layers/packages in my application. 

The proper solution highly depends on what the Go1..Go3 are. so what are they? What are you implementing?

 

1) There's in trigger in Go 1 that initiates a request for some data that is managed by Go 2. It (Go 1) sends a request with a key on reqData chan. 

2) Go 2 returns a pointer to the data on the data chan. 

3) Go 1 does some processing/updates to the data based on the trigger mentioned earlier, 

4) When complete, Go 1 sends a notification to Go 2 saying it's done processing the data, 

5) Go 2 does some of it's own processing on the data relevant to it's own layer.

6) Finally as one of those processing steps, Go 2 notifies Go 3 there was an update to the data by sending it (the updated data) on a chan to Go 3.

I would write the interaction piece... e.g.

func Interact(A A, B B, C C) error {

data, err := B.Request(A.GetParams())

if err != nil {

return err

}

err := A.HandleData(data)

if err != nil {

return err

}

B.DataHandled(data)

C.Notify(data)

}

And follow design of it from there...

By breaking up the interaction into different goroutines/objects/types you are making the interaction harder to understand.

I see what you're saying, and will look at how this could incorporate. Thanks for the different perspective.

 

 

My main concern is around the 3 channels I have between Go 1 and Go 2, because I cannot find anyone use this 3 chan approach, and I'm wondering if I'm making some larger mistake that could be solved a better way.

You might be overusing channels.

 

Yeah that's exactly my fear. I've been wondering if I could solve this with Mutexes instead, because that middle layer really spends time just protecting the data from concurrent access / data races.

 

Cheers,

Mark

[Jesper Louis Andersen]

On Sun, May 31, 2015 at 5:51 PM, Mark Laczynski <mjla...@gmail.com> wrote:

Go Routine 1 parses a continuous stream. That is its whole responsibility, because there's just a lot of logic in parsing the stream "packets."

Go Routine 2 manages and owns the data. So basically this routine is serializing the access to the data, because various parts of the application can be updating/reading from the data.

Go Routine 3 is an httpHandler that updates the UI. So Go Routine 2 notifies the UI to update the page with the new data.

Why is the work in GR1 concurrent with the work in GR2? GR3 and it's asynchronous updating is usually an elegant way.

Usually, when defining a concurrent system, you must define it's concurrent agents and explain why the work in one agent can happen concurrently with the work in the other agent. If you can't then chances are you are describing a sequential chain of events which means you should think about if you are better off coalescing the work into one goroutine. It may be there are more than one GR1 in the system, or that GR2 needs to remain responsive while the work is being processed, or something else along those lines.

A common trick you can use is to send around messages which contains channels. That way, the reply channel of GR1 can be sent along in the request and so on. It often simplifies code because processing becomes local to messages. It also handles multiplicity later on, since the data channel also acts like a tagging scheme for whom needs the data returned. Your system, with a globally known data channel, can't handle multiple GR1's later easily.

--

J.

[Mark Laczynski]

I think you're right. GR2 really isn't a go routine. The functionality it provides should actually be an API that is safe for concurrent access.

Thanks for the new perspective.

Cheers,
Mark