how to implement channel priorities?

[Erwin]

Hello,

i wonder how to implement channel priorities nicely. Say there are two channels, one with a higher priority than the other, and a goroutine waiting for incoming data on these channels. I have read that select picks a random case when multiple channels are ready. I thought of nesting selects: putting the lower priority select in the default case of the higher priority select, and have the default case of the inner (low priority) select do nothing. This leads to a kind of busy wait loop. I could call a short sleep, but that still isn't very clean. Is there a better way?

Why aren't the select cases evaluated in order?  

[Kyle Lemons]

This sounds like it might be a premature optimization, but you could do this in your loop:

select {

case val := <-high:

  continue

default:

}

select {

case val := <-high:

case val := <-low:

}

Having said that though, what problem are you trying to solve with "prioritized" channels?

[Jim Whitehead II]

They are. From the spec:

"RecvExpr must be a receive operation. For all the cases in the
"select" statement, the channel expressions are evaluated in
top-to-bottom order, along with any expressions that appear on the
right hand side of send statements. A channel may be nil, which is
equivalent to that case not being present in the select statement
except, if a send, its expression is still evaluated. If any of the
resulting operations can proceed, one of those is chosen and the
corresponding communication and statements are evaluated. Otherwise,
if there is a default case, that executes; if there is no default
case, the statement blocks until one of the communications can
complete. If there are no cases with non-nil channels, the statement
blocks forever. Even if the statement blocks, the channel and send
expressions are evaluated only once, upon entering the select
statement."

Not a particular answer to your question, but I'm wondering why you
think they're not..

- Jim

[Patrick Mylund Nielsen]

This got me too earlier. It's the evaluation of the cases, not which choice it picks.

"If multiple cases can proceed, a uniform pseudo-random choice is made to decide which single communication will execute."

[Erwin]

"RecvExpr must be a receive operation. For all the cases in the
"select" statement, the channel expressions are evaluated in
top-to-bottom order, along with any expressions that appear on the
right hand side of send statements. A channel may be nil, which is
equivalent to that case not being present in the select statement
except, if a send, its expression is still evaluated. If any of the
resulting operations can proceed, one of those is chosen and the
corresponding communication and statements are evaluated. Otherwise,
if there is a default case, that executes; if there is no default
case, the statement blocks until one of the communications can
complete. If there are no cases with non-nil channels, the statement
blocks forever. Even if the statement blocks, the channel and send
expressions are evaluated only once, upon entering the select
statement."

Hmm, i watched the Google I/O talk by Ron Pike, and seem to recall he said something along the lines of a "pseudo random" choice being made which of the "ready" channel cases is taken in a select... Also in the above definition: "If any of the resulting operations can proceed, one of those is chosen and the corresponding communication and statements are evaluated"... So ok, i should have been more precise and ask: why isn't the first select case taken that can proceed? 

[Erwin]

This sounds like it might be a premature optimization, but you could do this in your loop:

select {

case val := <-high:

  continue

default:

}

select {

case val := <-high:

case val := <-low:

}

Hmm, interesting, looks like this could work. 

 

Having said that though, what problem are you trying to solve with "prioritized" channels?

I am developing a control program to drive a set of networked video players (also programs) in a video wall. A player (server) is sent either "immediate mode" requests, or "retained mode" requests over the network. Now and then i want to switch to specific videos on all players as simultaneously as possible using immediate mode requests, and i would like that these immediate mode requests have higher priority than the retained mode requests,

in a situation that both are eligible to be sent.. just to save the time (and delay) of a retained mode request that is going to be overridden by an immediate mode request very shortly afterwards.. I hope that makes sense, it is about having the smoothest over-all switching possible.

[Erwin]

This sounds like it might be a premature optimization, but you could do this in your loop:

select {

case val := <-high:

  continue

default:

}

select {

case val := <-high:

case val := <-low:

}

Hmm, interesting, looks like this could work. 

On closer inspection, there is of course the possibility that right after the first select, in case "high" wasn't ready, things change, and both "high" and "low" are ready in the second select, enabling the choice of the lower priority channel.

I am not deep enough into the internals of Go to be sure this can happen, but still, it struck me as a potential flaw of this method. 

[Jim Whitehead II]

Assuming nothing about the scheduler (which you shouldn't do), we have
a situation here where a priority message could arrive immediately
after the first select. So when we reach the second select statement,
if both channels are ready, then a pseudo-random fair selection is
done to determine which message gets received.

The same sort of 'trick' is cited for Erlang as being their solution
to priority channels. The major difference is that in Erlang, there is
only one mailbox per process, and that is an ordered queue (although
later messages can be selected with pattern matching). Go has a much
nicer pattern here, since there's no 'out-of-order receiving'.

However, it isn't quite what you'd expect from a PRI ALT in occam
terms, however.

It would be quite interesting if you could alter the state of the
channels being listened to during a select statement. Something like:

low := make(chan int)
high := make(chan int)

select {
case val := <-high; low = nil:
// do the high priority thing here
low = origLow
case val := <-low:
// do the low priority thing here
}

Of course, this doesn't work and it's easy to see how something like
this could get incredibly messy =)

- Jim

[Kyle Lemons]

This method is not perfect, but it's as close as you can get with channels without implementing your own priority queue afaik.  It will prevent you from processing a new low priority message while there are existing high priority messages whenever you enter a loop.  As I said, it seems like it might be a bit of a premature optimization, as the chance that you will get conflicts (i.e. two channels becoming simultaneously ready) seems vanishingly small, at which point you're at the mercy of whichever arrives first.

[Vanja Pejovic]

Given a channel A that has a higher priority than channel B, it seems difficult to implement channels with priority such that it's impossible to process a request from B, while there is a request for be waiting. For example:

var A chan int

var B chan int

// set A to have higher priority than B, that is A is more important than B

// make the channels and send them to another goroutine

// assume that select implementation now first checks channels with high priority if they have pending messages and if they don't

// it waits until any channel is ready.

select { 

case val := <-A:

  process(val)

case val := <-B:

  process(val)

}

Now suppose there are no messages on A, and there is a ready message on B. The second case starts. Now suppose that this thread is pre-empted, and another thread puts a message on A and then gets pre-empted back to the original thread. Now we have a message waiting on A, and we really want to handle it first since it's much more important, but we are already in the second second case, and we have no choice.

The only thing I can think of is to make processing of values interuptable if a value with a higher priority comes in.

[Andrew Gerrand]

Here's something I tried, but it's not perfect:

http://play.golang.org/p/I8MS0DFgZ2

Andrew

[Ian Lance Taylor]

On Mon, Jul 23, 2012 at 5:29 PM, Andrew Gerrand <a...@golang.org> wrote:
> Here's something I tried, but it's not perfect:

As far as can see, there is no such thing as a perfect solution to
this problem unless you can constrain it in some way. It's always
possible that a value will arrive on the high priority channel at the
very instant that you decide to process the value on the low priority
channel.

Ian

[Jim Whitehead II]

That's a slightly different issue. What we're trying to prevent is the situation where there both high and low priority messages ready to be received and the system chooses the low priority one. The example I gave above would work for that, but would rely on the top-to-bottom ordering of select and would require that we allow additional expressions in the case statement. I'm not proposing we do this, but it does solve the problem being addressed.

- Jim

[Ingo Oeser]

In package container/heap is a documented example of a priority queue. Just dequeue all messages from the channel requeue them via the example code dequeue again and enqueue to a channel. This sounds complex but might be a more race free approach.

[N. Riesco - GMail]

Based on Andrew's idea, What about something like this?:

http://play.golang.org/p/uES_xmh0OA

-- Nico

[N. Riesco - GMail]

Never mind, I've just realised why Andrew used a loop.

-- Nico

[N. Riesco - GMail]

And here's another version, this time making use of len:

http://play.golang.org/p/xIEWTY8cTM

I'd be grateful, if people could point out any difficulties with this
version.

-- Nico

[Jim Whitehead II]

On Tue, Jul 24, 2012 at 11:06 AM, N. Riesco - GMail
<nicolas...@gmail.com> wrote:
> And here's another version, this time making use of len:
>
> http://play.golang.org/p/xIEWTY8cTM
>
> I'd be grateful, if people could point out any difficulties with this
> version.

I would say that using len() here actually slightly confuses things,
because that expression is only evaluated once (not each time the loop
goes around). This means that during both loops where you try to flush
any of the high priority messages, you only process as many as were
present when the loop started executing. If more arrive during the
processing of those, then they will not be covered.

In this case, Andrew's loop is a bit clearer since you know that it
will continue to process high priority messages until there are none
more available, whereas yours will go into another iteration of the
main loop even if there are messages waiting on the channel.

Subtle difference, but still =)

- Jim

[Chris Hines]

From the spec:

In its simplest form, a "for" statement specifies the repeated execution of a block as long as a boolean condition evaluates to true. The condition is evaluated before each iteration. If the condition is absent, it is equivalent to true. [emphasis mine]

So the len() expression is indeed evaluated on each loop.

A different concern would be potential race conditions because the len() expression is evaluated separately from the <-hi expression inside the loop.

Chris

On Tuesday, July 24, 2012 6:19:16 AM UTC-4, Jim Whitehead wrote:

[Jim Whitehead II]

On Tue, Jul 24, 2012 at 6:50 PM, Chris Hines <ggr...@cs-guy.com> wrote:
> From the spec:
>
> In its simplest form, a "for" statement specifies the repeated execution of
> a block as long as a boolean condition evaluates to true. The condition is
> evaluated before each iteration. If the condition is absent, it is
> equivalent to true. [emphasis mine]
>
>
> So the len() expression is indeed evaluated on each loop.
>
> A different concern would be potential race conditions because the len()
> expression is evaluated separately from the <-hi expression inside the loop.

I beg to differ:

http://play.golang.org/p/0w63yNxkRJ

The condition is indeed checked every single iteration. However, the
len(ch) is evaluated to a number at the start of the loop and that
never changes.

- Jim

[Chris Hines]

I'm not sure what your example proves. What about this?

http://play.golang.org/p/6BUQ-8xtV3

Clearly the loop would never exit if len() was not evaluated on each iteration.

Chris

On Tuesday, July 24, 2012 2:06:32 PM UTC-4, Jim Whitehead wrote:

[Jim Whitehead II]

On Tue, Jul 24, 2012 at 7:34 PM, Chris Hines <ggr...@cs-guy.com> wrote:
> I'm not sure what your example proves. What about this?
>
> http://play.golang.org/p/6BUQ-8xtV3
>
> Clearly the loop would never exit if len() was not evaluated on each
> iteration.

Yes, you're correct. I made a silly mistake, apologies for the noise!

- Jim

[Erwin]

if select chose its items in order, it would be impossible to determine, for a given select statement, whether or not all of those channel reads would eventually be reached.  That is, if you are selecting on two channels that are always ready, and it's non-random, only the first will ever be read from, and the second could be put into a state of isolation.  Resource starvation could occur if simultaneous channel-selection operations were not randomized.

Yes, that makes sense. 

 

I struggle to see why one would want to prioritize their channel reads, because it seems liable to give you concurrency bugs, but, either way, here is how you would do it:

    high, low := make(chan whatever), make(chan whatever)

    go someFunc(high)

    go someOtherFunc(low)

    if v, ok := <-high; ok {

        use(v)

    } else {

        if v, ok := <-low; ok {

            use(v)

        } else {

            // no value on either channel

        }

    }

 

Won't the above check for the channels being closed? The way i understand it, the lower priority channel would only be checked after the higher priority channel has been closed...

 

the concept of prioritizing channels makes my stomach uneasy.  What's the use case?

My use case is trying to get as accurate as possible synchronization of a number of goroutines that respond to two

types of channel commands, "immediate mode" and "retained mode". If an immediate mode command is available i would like to issue this first, no matter if a retained mode command is also available. The goroutines control devices that take a number of milliseconds to process a command, so i don't want to send a lower priority command and introduce latency, if a high priority command could be sent instead.

 

[Pete Wilson]

On Monday, July 23, 2012 4:52:03 PM UTC-5, Kyle Lemons wrote:

...

Having said that though, what problem are you trying to solve with "prioritized" channels?

The usual reason for prioritised selects is to handle the 'Big Red Button' situation, in which a process/go routine is accepting work form one or more channels, but also needs to handle a 'stop it NOW, dammit' message. This is very simply done by using a higher priority channel for such big red button messages.

It's why occam had 'PRI ALT' as well as 'ALT'

[roger peppe]

On 25 July 2012 00:41, Pete Wilson <pe...@kivadesigngroupe.com> wrote:
> It's why occam had 'PRI ALT' as well as 'ALT'

... although ALT was in fact implemented as PRI ALT.

[N. Riesco - GMail]

I have modified the example to address the potential race condition.
This version still uses for and len() instead of select as in Andrew's
solution:

http://play.golang.org/p/wWbIXbjdai

Nico

[N. Riesco - GMail]

This version can also potentially suffer from race conditions when
multiple goroutines read the hi channel. See:

https://groups.google.com/d/msg/golang-nuts/n6PIY22hbYI/90_8BUkcVDEJ

and

https://groups.google.com/d/msg/golang-nuts/n6PIY22hbYI/PQH6GE147C8J

for an explanation.

Nico

[Steven Blenkinsop]

I think it's important to remember that, when you're dealing with concurrency, there is an amount of indeterminacy. In the cases where the value you're going to receive has already been sent before you start trying to receive, the two-select approach suggested at the beginning of this thread does the right thing, usually. The one exception is that it might receive the high priority value even when it arrives after you begin, which seems like a plus.

Once you're expecting values to arrive while you're receiving, whether one value arrives before the other – or before some arbitrary point in the receive operation – is really indeteminate. Thinking about it terms of what happens first isn't very useful. The cases where something non-ideal could happen are fairly unlikely and penalize both priorities equally. Since the order the values arrive in is already arbitrary at this point, it seems like the two-select approach should be an acceptable solution.

[N. Riesco - GMail]

Yes, I agree. I think the two-select approach does the right thing. I
can now see how select protects against a series of potential race
conditions.

Nico

On 25/07/12 18:11, Steven Blenkinsop wrote:
> I think it's important to remember that, when you're dealing with
> concurrency, there is an amount of indeterminacy. In the cases where the
> value you're going to receive has already been sent before you start
> trying to receive, the two-select approach suggested at the beginning of
> this thread does the right thing, usually. The one exception is that it
> might receive the high priority value even when it arrives after you
> begin, which seems like a plus.
>
> Once you're expecting values to arrive while you're receiving, whether

> one value arrives before the other � or before some arbitrary point in
> the receive operation � is really indeteminate. Thinking about it terms

[kang...@gmail.com]

select {

case highVal := <- high:

case lowVal := <- low:

        if len(high) > 0 {

            for len(high) > 0 {

                highVal := <- high

            }

        }       

        // process lowVal

}

On Tuesday, July 24, 2012 at 5:16:16 AM UTC+8, Erwin Driessens wrote:

Hello,

i wonder how to implement channel priorities nicely. Say there are two channels, one with a higher priority than the other, and a goroutine waiting for incoming data on these channels. I have read that select picks a random case when multiple channels are ready. I thought of nesting selects: putting the lower priority select in the default case of the higher priority select, and have the default case of the inner (low priority) select do nothing. This leads to a kind of busy wait loop. I could call a short sleep, but that still isn't very clean. Is there a better way?

Why aren't the select cases evaluated in order?  

[Anthony Metzidis]

Fun thought exercise -- here's another approach.

https://play.golang.org/p/Xu7iWhY4PUQ

package main

import (

"fmt"

"math/rand"

"sort"

"time"

"strconv"

)

type PriorityStruct struct {

P    int

name string

}

const BUFSIZE = 10

type PriorityChan chan PriorityStruct

type PriorityBuf []PriorityStruct

func (s PriorityBuf) Len() int {

return len(s)

}

func (s PriorityBuf) Swap(i, j int) {

s[i], s[j] = s[j], s[i]

}

func (s PriorityBuf) Less(i, j int) bool {

return s[i].P < s[j].P

}

func Sender(pchan PriorityChan) {

for {

randNum := rand.Intn(3)

pchan <- PriorityStruct{

P:    randNum,

name: "Bob-" + strconv.Itoa(randNum),

}

time.Sleep(10 * time.Millisecond)

}

}

func Prioritize(inChan PriorityChan, outChan PriorityChan) {

for {

buf := make(PriorityBuf, BUFSIZE)

for i, _ := range buf {

buf[i] = <-inChan

}

sort.Sort(buf)

for i, _ := range buf {

outChan <- buf[i]

}

}

}

func Receiver(rchan PriorityChan) {

for {

fmt.Printf("%v\n", <-rchan)

}

}

func main() {

inChan := make(PriorityChan)

outChan := make(PriorityChan)

go Sender(inChan)

go Receiver(outChan)

go Prioritize(inChan, outChan)

<- time.After(5 * time.Second)

}

 

[roger peppe]

On Thu, 23 May 2019 at 01:34, Anthony Metzidis <anthony....@gmail.com> wrote:

Fun thought exercise -- here's another approach.

https://play.golang.org/p/Xu7iWhY4PUQ

ISTM that that approach doesn't work because you're buffering 10 values before you send any.

So if you have a situation when you've got very little traffic on all the channels, a message might

get delayed indefinitely.

For example, here's your example with only one message being sent - it hasn't arrived

at the receiver after 5 seconds have elapsed.

  cheers,

    rog.

[roger peppe]

Oops, forgot to include the playground link at the end there: https://play.golang.org/p/mYSRsGb4mRA

[Bruno Albuquerque]

This was my attempt at a channel with priorities:

https://git.bug-br.org.br/bga/channels/src/master/priority/channel_adapter.go

Based on the assumption that priorities only make sense if items are queued. If they are pulled out from a channel as fast as they are added to it, then there is no need for priorization.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAJhgaciGHN633%2BCWtYE6Gw4i8pdB9XoQ5HbOiHR1X06oJxMdCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Anthony Metzidis]

Good point i'll update with a timeout to flush in case traffic is slow

On Thu, May 23, 2019 at 1:38 AM roger peppe <rogp...@gmail.com> wrote:

[roger peppe]

On Thu, 23 May 2019 at 19:28, Bruno Albuquerque <b...@gmail.com> wrote:

This was my attempt at a channel with priorities:

https://git.bug-br.org.br/bga/channels/src/master/priority/channel_adapter.go

Based on the assumption that priorities only make sense if items are queued. If they are pulled out from a channel as fast as they are added to it, then there is no need for priorization.

I'm not entirely sure that's the right assumption. The original request is about having priority

between different channels. If there's a value available on both channels, you only ever want to

process the high priority one. With your code, however, if you've got two goroutines sending and the

output channel is always ready to receive, the priority is decided by goroutine scheduling, and

the low priority channel can win inappropriately: https://play.golang.org/p/YBD_w5vVqjt

There are other issues too: the goroutine effectively acts as an infinite buffer, so any back pressure

from the output channel is lost; if you have a slow receiver, you could use a large amount of memory.

Also, if the input channel is closed, your code will discard any values in the buffer: https://play.golang.org/p/5e_E17klnef

You perhaps want something a bit more like this: https://play.golang.org/p/hPHSRvpsqxa

FWIW the classical way to address the original problem in Go AIUI is to select on the

high priority channels first (with a default clause), and then block on all the channels if there's

nothing immediately ready. This idiom is a bit tedious to generalise to multiple channels (first select on c1, then c1, c2, then c1, c2, c3,

etc until finally select on c1, ... cN), but isn't two bad with two.

     https://play.golang.org/p/pkgZsgENdDb

  cheers,

    rog.

[roger peppe]

On Fri, 24 May 2019 at 16:25, Bruno Albuquerque <b...@gmail.com> wrote:

On Fri, May 24, 2019 at 1:50 AM roger peppe <rogp...@gmail.com> wrote:

On Thu, 23 May 2019 at 19:28, Bruno Albuquerque <b...@gmail.com> wrote:

This was my attempt at a channel with priorities:

https://git.bug-br.org.br/bga/channels/src/master/priority/channel_adapter.go

Based on the assumption that priorities only make sense if items are queued. If they are pulled out from a channel as fast as they are added to it, then there is no need for priorization.

I'm not entirely sure that's the right assumption. The original request is about having priority

between different channels. If there's a value available on both channels, you only ever want to

process the high priority one. With your code, however, if you've got two goroutines sending and the

output channel is always ready to receive, the priority is decided by goroutine scheduling, and

the low priority channel can win inappropriately: https://play.golang.org/p/YBD_w5vVqjt

My code actually uses a single channel and deals with relative priorities between items added to the channel. The original problem can be morphed to this by using only 2 priorities. I honestly do not think that worrying about 2 values being sent by 2 different go routines at nearly the same time is something one needs to worry about. This is similar to having 2 entries sent to individual channels at nearly the same time but with enough skew that the low priority one arrives before the higher priority one does (so it will be processed first unless there is a forced delay added (as seem on the other approach). 

One example scenario when you might want to have priority receiving on two different channels (which was after all the question that started this thread), is when you're receiving messages from a high-volume buffered channel, and a low volume synchronous channel (for example, a channel returned from context.Done). If you get a value from the Done channel, you want to quit as soon as possible. If you just use a regular select, you can end up processing an unbounded number of messages before you decide to quit.

There is one big advantage with my solution: You can actually model any reasonable number of priorities you might need without any code changes whatsoever. It can handle 2 priorities as well as it can handle 100.

If I may say so, there's one significant disadvantage of your solution: it doesn't solve the original problem :)

  cheers,

    rog.

[Bruno Albuquerque]

On Fri, May 24, 2019 at 1:50 AM roger peppe <rogp...@gmail.com> wrote:

On Thu, 23 May 2019 at 19:28, Bruno Albuquerque <b...@gmail.com> wrote:

This was my attempt at a channel with priorities:

https://git.bug-br.org.br/bga/channels/src/master/priority/channel_adapter.go

Based on the assumption that priorities only make sense if items are queued. If they are pulled out from a channel as fast as they are added to it, then there is no need for priorization.

I'm not entirely sure that's the right assumption. The original request is about having priority

between different channels. If there's a value available on both channels, you only ever want to

process the high priority one. With your code, however, if you've got two goroutines sending and the

output channel is always ready to receive, the priority is decided by goroutine scheduling, and

the low priority channel can win inappropriately: https://play.golang.org/p/YBD_w5vVqjt

My code actually uses a single channel and deals with relative priorities between items added to the channel. The original problem can be morphed to this by using only 2 priorities. I honestly do not think that worrying about 2 values being sent by 2 different go routines at nearly the same time is something one needs to worry about. This is similar to having 2 entries sent to individual channels at nearly the same time but with enough skew that the low priority one arrives before the higher priority one does (so it will be processed first unless there is a forced delay added (as seem on the other approach). 

 

There are other issues too: the goroutine effectively acts as an infinite buffer, so any back pressure

from the output channel is lost; if you have a slow receiver, you could use a large amount of memory.

Also, if the input channel is closed, your code will discard any values in the buffer: https://play.golang.org/p/5e_E17klnef

You perhaps want something a bit more like this: https://play.golang.org/p/hPHSRvpsqxa

This was more a proof of concept than anything else. It would be trivial to limit the number of inflight items at the cost off blocking on the input channel. It is also trivial to empty the buffer before closing the output channel (thanks for the suggestion).

 

FWIW the classical way to address the original problem in Go AIUI is to select on the

high priority channels first (with a default clause), and then block on all the channels if there's

nothing immediately ready. This idiom is a bit tedious to generalise to multiple channels (first select on c1, then c1, c2, then c1, c2, c3,

etc until finally select on c1, ... cN), but isn't two bad with two.

     https://play.golang.org/p/pkgZsgENdDb

There is one big advantage with my solution: You can actually model any reasonable number of priorities you might need without any code changes whatsoever. It can handle 2 priorities as well as it can handle 100. In fact if somehow if you end up with a lot of queued items at priority 100 but you really need something else to be processed immediately, you can send it at priority 101 and that will work as expected.

[Bruno Albuquerque]

On Fri, May 24, 2019 at 8:40 AM roger peppe <rogp...@gmail.com> wrote:

On Fri, 24 May 2019 at 16:25, Bruno Albuquerque <b...@gmail.com> wrote:

On Fri, May 24, 2019 at 1:50 AM roger peppe <rogp...@gmail.com> wrote:

On Thu, 23 May 2019 at 19:28, Bruno Albuquerque <b...@gmail.com> wrote:

This was my attempt at a channel with priorities:

https://git.bug-br.org.br/bga/channels/src/master/priority/channel_adapter.go

Based on the assumption that priorities only make sense if items are queued. If they are pulled out from a channel as fast as they are added to it, then there is no need for priorization.

I'm not entirely sure that's the right assumption. The original request is about having priority

between different channels. If there's a value available on both channels, you only ever want to

process the high priority one. With your code, however, if you've got two goroutines sending and the

output channel is always ready to receive, the priority is decided by goroutine scheduling, and

the low priority channel can win inappropriately: https://play.golang.org/p/YBD_w5vVqjt

My code actually uses a single channel and deals with relative priorities between items added to the channel. The original problem can be morphed to this by using only 2 priorities. I honestly do not think that worrying about 2 values being sent by 2 different go routines at nearly the same time is something one needs to worry about. This is similar to having 2 entries sent to individual channels at nearly the same time but with enough skew that the low priority one arrives before the higher priority one does (so it will be processed first unless there is a forced delay added (as seem on the other approach). 

One example scenario when you might want to have priority receiving on two different channels (which was after all the question that started this thread), is when you're receiving messages from a high-volume buffered channel, and a low volume synchronous channel (for example, a channel returned from context.Done). If you get a value from the Done channel, you want to quit as soon as possible. If you just use a regular select, you can end up processing an unbounded number of messages before you decide to quit.

And you could easily do that with a select on the channel I return and the done channel. Also this is a very specific scenario you propose. For specific scenarios one can almost always find specific non-generic solutions that are better than a generic one. My solution tries to address the general case of dealing with prioritizing items in a channel.

 

There is one big advantage with my solution: You can actually model any reasonable number of priorities you might need without any code changes whatsoever. It can handle 2 priorities as well as it can handle 100.

If I may say so, there's one significant disadvantage of your solution: it doesn't solve the original problem :)

You you take the original question literally, you are obvious right. But it the original question can be relaxed to "I want to handle items with different priorities" which, as I mentioned, the original problem can be reduced to, then no, you are not.

Anyway, I am fine with agreeing to disagree.

 

[Michael Jones]

I see an easy* way to implement channel priorities in Go2 if that is desired. 

two steps:

step 2: At lines 124 and 153-157 of src/runtime/select.go, an order of channel testing index table is created, then permuted. 

It is later visited in order. All that needs doing is to make the index table larger by repeating channel indices based on relative priority. example:

channels a,b,c of equal priority:

pollorder = permute[a,b,c]

channels a,b,c of priority a=5,b=2,c=1:

pollorder = permute[a,a,a,a,a b,b, c]

step 1:Somehow convey priorities from developer's Go source to select.go. Open issue, many choices, matter of taste. One idea is explicit parameter in select, another is to allow cases to repeat and count them, or equivalently, allow the "comma list of channels" with repeats.

 select {

        case msg1 := <-c1:

            fmt.Println("received", msg1)

        case msg1 := <-c1:

            fmt.Println("received", msg1)

        case msg2 := <-c2:

            fmt.Println("received", msg2)

 }

 select {

        case msg1 := <-c1, <-c1:

            fmt.Println("received", msg1)

        case msg2 := <-c2:

            fmt.Println("received", msg2)

 }

Each of these means c1 is 2x and c2 is 1x out of a total of 3, for c1 is 2x the priority of c2 and is selected 2/3 of the time on average when both channels have data. Since a deranged developer might copy/paste the "<-c1" a hundred times, and "<-c2" fifty, the compiler should preferably optimize:

g = gcd(list of priorities)

[list of priorities]/=g // defend against needless scales in priorities, ivy-like syntax

Michael

*easy to conjecture.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEd86Tym8dvnBHQj6Oudyc_K31-HgG-s4cQjB%3DbNzSpX11HKtw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Michael T. Jones
michae...@gmail.com

[Daniela Petruzalek]

If I had to process messages from both high and low priority channels I would serialise then with a min heap and create a consumer process for the min heap. The min heap seems to be the canonical way of solving the priority queues anyway, so I think it makes sense here.

Basically: goroutine 1 reads both channels and insert them into the min-heap. goroutine 2 reads from min heap and do the processing.

Daniela Petruzalek

Software Engineer

github.com/danicat

twitter.com/danicat83

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Michael Jones]

Daniela, that seems an excellent solution.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CA%2B%3Diygt2bxH1E8fkD5JpPV1vKrz2ns%2BD5KpbhZ_SSm2vK%3Dds3w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Bruno Albuquerque]

This is what my solution does, except that it uses items associated with priorities instead of 2 channels. if 2 channels are really needed for some reason, it is easy to adapt the solution to this.

 

On Fri, May 24, 2019 at 12:12 PM Daniela Petruzalek <daniela.p...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CA%2B%3Diygt2bxH1E8fkD5JpPV1vKrz2ns%2BD5KpbhZ_SSm2vK%3Dds3w%40mail.gmail.com.