Recovery for RingBuffer

[rama atluri]

I have my Ring Buffer set up as below

 

Disruptor incomingDisruptor = new Disruptor<T> (eventFactory, 1024, EXECUTOR, ProducerType.MULTI, waitStrategy)

BatchEventProcessor batchDBProcessor = new BatchEventProcessor(incomingDisruptor.getRingBuffer(), incomingDisruptor.getRingBuffer().newBarrier(), dbJournal);

EventHandlerGroup eventGroup = incomingDisruptor.handleEventsWith(batchDBProcessor);

EventHandlerGroup eventGroup1 = eventGroup.then(messageParser);

eventGroup1.then(businessHandler);

as part of cold restart procedures of the incomingDisruptor

DB Journaler is set at 951 seq no ; buiness Handler is at 751 seq no;

so we try to set the incoming disruptor to start publihing events from 751 sequence and replay/publish the messages from 751 to 951.

the code would then become

incomingDisruptor.getRingBuffer().resetTo(751)

batchDBProcessor.getSequence().set(951)

we then start publising events to the incoming buffer from 751 to 951.

expected result is that DB journaler to receive events after 951 and messageparser/buiness handler from 751.

what happens we see DB journaler receiving events after 951 and NO events received for messageparsers & buiness handlers ?

Looks like some thing wrong with the way the disruptor event handlers are set up. Is this b'cos there two Sequence barriers created one for messageParser and other for business handler

If i comment out businessHandler. then I could see the events firing to message Parser.

any comments on this is appreciated.

thank you.

 

 

 

 

[Michael Barker]

What sequence do you set the message parser/business handler to?

[rama atluri]

Hi Mike,

 

batchDBProcessor is a BatchEventProcessor. I can set the batchDBProcessor to start processing events from specific seq no by using the method batchDBProcessor.getSequence().set(951).

With  messageParser & businessHandler (implement interface EventHandler) don't have access sequence barrier to specify a start seq no. How can we set up EventHandler's to start receiving events from a sequence number.

This would be helpful in recovery scenarios where handlers to start consuming events from a sequence no.

 

The order of set up

 

batchDBProcessor -> parser/validator -> businessHandler

[Michael Barker]

Hi Rama,

Where I need to do this, I usually do
EventProcessor.getSequence().set(...), unfortunately with the
Disruptor DSL you don't always have a handle on the EventProcessor.
However, in your case I'm wondering if you could avoid it completely.
If you stopped using the sequence number from the Disruptor and
instead created your own sequence value. The recovery would just
involve replaying any unprocessed events back through the Disruptor.
If you didn't have a mechanism to manage your own sequence then you
could do something along the lines of holding an offset in each of the
event handlers. At start up find the last completely processed event
and set the offset to be that value plus one. Then your application
sequence becomes the Disruptor sequence plus the offset. That would
remove a lot of the complexity around resetting the sequence numbers
in the Disruptor.

We do a degree of sequence resetting in our application, but it is
ugly and I wouldn't recommend doing it. I would look for an
implementation where you don't need to reset the sequences.

Mike.

> --
> You received this message because you are subscribed to the Google Groups
> "Disruptor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to lmax-disrupto...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[rama atluri]

Thanks Mike for your response.

 

Agree to  leave sequence handling to Ring Buffer. We will try to implement the recovery procedures around it.

 

On Wednesday, May 15, 2013 4:31:46 PM UTC+10, rama atluri wrote: