Disruptor
Christoph Hack
John Arbash Meinel
Hash: SHA1
On 9/22/2011 3:57 AM, Christoph Hack wrote:
> Hi folks,
>
> fluffle has recently send me a link about the LMAX Architecture
> <http://martinfowler.com/articles/lmax.html>, a quite exceptional
> concurrent architecture which main goal was to avoid
> synchronization primitives and concurrency as much as possible.
> Their excellent technical paper
> <http://disruptor.googlecode.com/files/Disruptor-1.0.pdf> also
> contains a good introduction about CPU caches in general.
>
> After reading their paper, I immediately wanted to try out those
> things I just have learned. My goal was to transfer a lot of
> integers between two goroutines, which basically is a simple
> 1-producer + 1-consumer topology. Please note, that the Disruptor
> <http://code.google.com/p/disruptor/> design can be easily
> scaled-up to many-producers + many-consumers, although with sightly
> different semantics than the (universally usable) Go channels.
>
> Anyway, some unscientific micro-benchmarks revealed the following
> result:
>
> Channels: 173 ns/op Disruptor (Go): 38.4 ns/op Disruptor (C++):
> 6.03 ns/op
>
One thing to think about. If you are passing a lot of data over
channels, you can just as easily pass arrays of data over channels,
and then work on the content in a loop. So instead of:
func producer(out chan int) {
for i := range 10000 {
// Do real stuff here
out <- i;
}
}
you can do:
const BUF_SIZE = 100
func producer(out chan []int) {
buf := make([]int, BUF_SIZE, BUF_SIZE)
offset = 0
for i := range 10000 {
if offset == BUF_SIZE {
out <- buf
buf := make([]int, BUF_SIZE, BUF_SIZE)
offset = 0
}
// Do real stuff here
buf[offset] = i
offset++
}
}
I know your code does allow buffering in the channel, but you'd be
better off buffering into an array if you know that you are going to
be producing batches.
Also, your low level operation is a busy loop. Which looks great when
you have 2 of them and 2 threads and 2 cpus. But what happens if you
wanted to have 100 'channels'. Note that in Go, goroutines are
cooperative multi-taskers (not preemptive). So if you busy-wait on a
coroutine that is multiplexed with another coroutine, you could deadlock.
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk57Bl4ACgkQJdeBCYSNAAPqbACfYU/a0/1lBBCW1qOG44mVJ9P4
N0IAn2MS6ebPiL+arXiHVlEraaUq5fRt
=3CG6
-----END PGP SIGNATURE-----
tiny_dust
will be better.
