Unbounded FIFO
493 views
Skip to first unread message
Igor Gatis
Feb 16, 2014, 5:32:24 AM2/16/14
to golan...@googlegroups.com
What's the idiomatic way of having a unbounded FIFO mechanism in go?
It seems I need to define it myself, perhaps using a channel as the locker.
Dmitry Vyukov
Feb 16, 2014, 5:54:41 AM2/16/14
to Igor Gatis, golang-nuts
If you mean unbounded producer/consumer FIFO channels, there is no
idiomatic way, because there are no machines with unbounded memory. So
you always want a bounded channel.
On Sun, Feb 16, 2014 at 2:32 PM, Igor Gatis <igor...@gmail.com> wrote:
> What's the idiomatic way of having a unbounded FIFO mechanism in go?
>
> It seems I need to define it myself, perhaps using a channel as the locker.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
idiomatic way, because there are no machines with unbounded memory. So
you always want a bounded channel.
On Sun, Feb 16, 2014 at 2:32 PM, Igor Gatis <igor...@gmail.com> wrote:
> What's the idiomatic way of having a unbounded FIFO mechanism in go?
>
> It seems I need to define it myself, perhaps using a channel as the locker.
>
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
Igor Gatis
Feb 16, 2014, 6:05:18 AM2/16/14
to Dmitry Vyukov, golang-nuts
Ok, so if I don't know upfront the size a channel needs to be, what should I do?
Dmitry Vyukov
Feb 16, 2014, 6:30:15 AM2/16/14
to Igor Gatis, golang-nuts
If you have, say, 1GB of memory and each request consumes 1K, you know
that channel size must be no larger than 1e6. Otherwise the process
will either die immediately (good case) or cause swapping (bad case).
Also you need buffering only to (1) achieve better performance and (2)
handle spikes. Channel of size 1000-10000 is usually enough for this.
There is no reason to see how the channel grows to 1e6 until you start
throttle. If a system steadily can't keep up with load, you have to
drop, nack, ask to resend later, flush to db or whatever.
There is no magical way how the system can sustain infinite number of
request and free you from solving the overload problem.
that channel size must be no larger than 1e6. Otherwise the process
will either die immediately (good case) or cause swapping (bad case).
Also you need buffering only to (1) achieve better performance and (2)
handle spikes. Channel of size 1000-10000 is usually enough for this.
There is no reason to see how the channel grows to 1e6 until you start
throttle. If a system steadily can't keep up with load, you have to
drop, nack, ask to resend later, flush to db or whatever.
There is no magical way how the system can sustain infinite number of
request and free you from solving the overload problem.
Jan Mercl
Feb 16, 2014, 6:30:44 AM2/16/14
to Igor Gatis, Dmitry Vyukov, golang-nuts
On Sun, Feb 16, 2014 at 12:05 PM, Igor Gatis <igor...@gmail.com> wrote:
> Ok, so if I don't know upfront the size a channel needs to be, what should I
> do?
If the producing side is on average slower or of same speed as the
> Ok, so if I don't know upfront the size a channel needs to be, what should I
> do?
consumer side then all you need is a buffer of the worst case burst
size. if the producing side is on average faster than the consumer
side than nothing can help if you the producer cannot block. If it can
block then any buffer size will fit.
-j
Reply all
Reply to author
Forward
0 new messages