Re: How to use a disruptor with multiple message types?
1,597 views
Skip to first unread message
Michael Barker
May 30, 2013, 6:38:50 PM5/30/13
to lmax-di...@googlegroups.com
Hi Ben,
Answers inline.
> Configure the disruptor to use objects containing a fixed size byte array
> (as recommended by How should one use Disruptor (Disruptor Pattern) to build
> real-world message systems?). In this case, the main thread must encode the
> messages into byte arrays before publishing them to the disruptor and each
> of the business logic threads must decode the byte arrays back into objects
> upon receipt. The downside of this setup is that the business logic threads
> are not truly sharing the memory from the disruptor - instead they are
> creating new objects (and thus creating garbage) from the byte array
> provided by the disruptor. The upside of this setup is that all business
> logic threads can read multiple different types of messages from the same
> disruptor.
This would be my preferred approach, but I slightly coloured by our
use cases, just about every place that we've used the Disruptor it's
either receiving from or sending to some sort of I/O device, so our
basic currency is byte arrays. You can get around the object creation
by using a flyweight approach to marshalling. To see an example of
this, I used Javolution's Struct and Union classes in an example that
I presented at Devoxx (https://github.com/mikeb01/ticketing). If you
can completely deal with the object before returning from the onEvent
call from the event handler then this approach works well. If the
event needs to live beyond that point then you need to make some sort
of copy of the data, e.g. de-serialising it into an object.
> Configure the disruptor to use a single type of object but create multiple
> disruptors, one for for each object type. In the case above, there would be
> two separate disruptors - one for objects of type A and another for objects
> of type B. The upside of this setup is that the main thread doesn't have to
> encode the object to a byte array and the business less logic threads can
> share the same objects as used in the disruptor (no garbage created). The
> downside of this setup is that somehow each business logic thread will have
> to subscribe to messages from multiple disruptors.
Not tried this approach, you'd probably need a custom EventProcessor
that can poll from multiple ring buffers.
> Configure the disruptor to use a single type of "super" object that contains
> all fields of both message A and B. This is very against OO style, but will
> allow for a compromise between option #1 and #2.
> Configure the disruptor to use object references. However, in this case I
> lose the performance benefits of object preallocation and memory ordering.
We've done this in a couple of cases where some cases where lack of
preallocation is tolerable. It works okay. If you are passing
objects then you need to make sure that you null them out once you are
finished with them on the consumer side. We found that using a double
dispatch pattern for the "super" object kept the implementation fairly
clean. One drawback to this is that it you will get slightly longer
GC stalls that with something that was a straight array of objects as
the GC has more live objects to traverse during the mark phase.
> What do you recommend for this situation? I feel that option #2 is the
> cleanest solution, but I don't know whether or how consumers can technically
> subscribe to messages from multiple disruptors. If anyone can provide an
> example for how to implement option #2, it would be much appreciated!
Another option, if you want complete flexibility with regards to the
use of data, is to not use the ring buffer, but instead talk directly
to the Sequencer and define your object layout as you best see
fitting.
Mike.
Answers inline.
> Configure the disruptor to use objects containing a fixed size byte array
> (as recommended by How should one use Disruptor (Disruptor Pattern) to build
> real-world message systems?). In this case, the main thread must encode the
> messages into byte arrays before publishing them to the disruptor and each
> of the business logic threads must decode the byte arrays back into objects
> upon receipt. The downside of this setup is that the business logic threads
> are not truly sharing the memory from the disruptor - instead they are
> creating new objects (and thus creating garbage) from the byte array
> provided by the disruptor. The upside of this setup is that all business
> logic threads can read multiple different types of messages from the same
> disruptor.
This would be my preferred approach, but I slightly coloured by our
use cases, just about every place that we've used the Disruptor it's
either receiving from or sending to some sort of I/O device, so our
basic currency is byte arrays. You can get around the object creation
by using a flyweight approach to marshalling. To see an example of
this, I used Javolution's Struct and Union classes in an example that
I presented at Devoxx (https://github.com/mikeb01/ticketing). If you
can completely deal with the object before returning from the onEvent
call from the event handler then this approach works well. If the
event needs to live beyond that point then you need to make some sort
of copy of the data, e.g. de-serialising it into an object.
> Configure the disruptor to use a single type of object but create multiple
> disruptors, one for for each object type. In the case above, there would be
> two separate disruptors - one for objects of type A and another for objects
> of type B. The upside of this setup is that the main thread doesn't have to
> encode the object to a byte array and the business less logic threads can
> share the same objects as used in the disruptor (no garbage created). The
> downside of this setup is that somehow each business logic thread will have
> to subscribe to messages from multiple disruptors.
Not tried this approach, you'd probably need a custom EventProcessor
that can poll from multiple ring buffers.
> Configure the disruptor to use a single type of "super" object that contains
> all fields of both message A and B. This is very against OO style, but will
> allow for a compromise between option #1 and #2.
> Configure the disruptor to use object references. However, in this case I
> lose the performance benefits of object preallocation and memory ordering.
We've done this in a couple of cases where some cases where lack of
preallocation is tolerable. It works okay. If you are passing
objects then you need to make sure that you null them out once you are
finished with them on the consumer side. We found that using a double
dispatch pattern for the "super" object kept the implementation fairly
clean. One drawback to this is that it you will get slightly longer
GC stalls that with something that was a straight array of objects as
the GC has more live objects to traverse during the mark phase.
> What do you recommend for this situation? I feel that option #2 is the
> cleanest solution, but I don't know whether or how consumers can technically
> subscribe to messages from multiple disruptors. If anyone can provide an
> example for how to implement option #2, it would be much appreciated!
Another option, if you want complete flexibility with regards to the
use of data, is to not use the ring buffer, but instead talk directly
to the Sequencer and define your object layout as you best see
fitting.
Mike.
ben.ba...@gmail.com
May 30, 2013, 7:55:53 PM5/30/13
to lmax-di...@googlegroups.com
Wow, thank you for the quick and detailed response! I will investigate using Javolution as a flyweight and see how it goes.
In case anyone else is looking for it, the link to video of your Devoxx presentation that goes along with the ticketing project that you referenced is below:
Thanks,
Ben
Michael Barker
May 31, 2013, 6:31:56 PM5/31/13
to lmax-di...@googlegroups.com
You're correct there is a race. However, fixing it did not add
anything with regards to explaining how to use the Disruptor.
Mike.
On 1 June 2013 08:43, <da...@vast.com> wrote:
> Is the polling logic in ResponseServlet correct, or only a rough demo?
> Because when I walk through it, there seems to be a race that send to the
> client an section_updated event for a concert the client hasn't seen, and
> it's not clear to me how to fix it.
>
> Not really a ringBuffer question. Sorry about that.
> --
> You received this message because you are subscribed to the Google Groups
> "Disruptor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to lmax-disrupto...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
anything with regards to explaining how to use the Disruptor.
Mike.
On 1 June 2013 08:43, <da...@vast.com> wrote:
> Is the polling logic in ResponseServlet correct, or only a rough demo?
> Because when I walk through it, there seems to be a race that send to the
> client an section_updated event for a concert the client hasn't seen, and
> it's not clear to me how to fix it.
>
> Not really a ringBuffer question. Sorry about that.
> You received this message because you are subscribed to the Google Groups
> "Disruptor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to lmax-disrupto...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
Michael Barker
May 31, 2013, 6:32:28 PM5/31/13
to lmax-di...@googlegroups.com
It's an exercise for the reader :-).
Reply all
Reply to author
Forward
0 new messages