What do I choose- Recovery based on Replay Params or Snapshot ?- Java API.

[Sathish]

HI All,

            I am trying to use Eventsourced to develop a POC to potentially build a Actor based listeners/subscriptions in a traditional J2EE based application.

This is what the framework does today:

1. We have REST based services which invoke Business Services and they invoke DAO Entitities to respond to certain read/write requests of the application.

2. No we have been trying to send certain events that happen inside the Business Services to External Applications.

3. Currently , we have all the JMS dependencies cooked inside the Business services, so any time some event needs to be fed to an external system , we add

    a new JMS Queue and add dependencies to the Business service layer after its processing.

4. We really want to use an Event Bus that will help us with coupling the dependency of JMS in the Business Services layer.

5. I have been searching for a lot of ways to Implement it - Guava Event Bus , AXON CQRS Framework and now I am onto Eventsourced.

6.I think that Eventsourced gets me 90% to where I want to be , but I some confusion when it comes to the recovery part of it.

We are trying to achieve persistence of every event that needs to be fed to an external system. First off , The ReliableChannel does the job with my use case.

But here is something that either that I dont follow or unable to get from the Documentation:

1. An event is published to the Processor , the processor then persists it and sends it off to the final destination.

2. If the final destination doesnt receive this message , the processor tries to send it first when the next event comes in.

3. What I really happening is , the processor is replaying all the events that it has received in its life time.

4. Now , the ReplayParams could be of help here was what I thought - Create a replay param with the last successful SequenceNr and use it as a fromSequenceNr - and I will good to go. Is this thought correct ?

5. How will you get the last successful Sequence Nr , How will the published know that when the all that the final destination knows is about the processor?

6. It looks like the there is no API around getting the last generated sequence Number.

7. Since I cannot get the last generated sequence Number , should I use the Snapshot Recovery approach , where a snapshot gets created after every processor.tell method call ?

Below is a very rough diagram with the idea , Can any one please advice ?

Any response to this is deemed as help :)

Thanks,

Sathish.

[Martin Krasser]

Hi Sathish,

are you only interested in a) reliable message delivery or also b) persistence of stateful actors? If it's only a) I recommend using a message broker of your choice (and hide its interfaces behind any abstraction of your choice). If it's also b) then Eventsourced might be an option, using reliable channels to connect persistent/eventsourced actors to a message broker (that manages the pub/sub channels).

Cheers,
Martin

--
You received this message because you are subscribed to the Google Groups "Eventsourced User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eventsourced...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
Martin Krasser

blog:    http://krasserm.blogspot.com
code:    http://github.com/krasserm
twitter: http://twitter.com/mrt1nz

[Sathish]

Hi Martin,

                   Thank you for your reply - I think I am looking for choice b and hence i thought I was on the right track by being on this forums.

Thank you for your reply. 

Can you please look at my questions related to the Replay Params ( based on the last SuccessfulSequence Number). Can you please help me with the questions that I have got and I posted in my prior message ?

Now that Eventsourced is becoming Akka-Persistence , can we ask for something like eventExtension.recover(replayParams.lastSuccessfulSequenceNr) ? 

Once again thank you for looking into the thread.

Thanks,

Sathish.

[HP]

Hi Satish, I can attempt to answer the question with my limited understanding of Eventsourced.  First, I am trying to understand your use-case here.  The behavior you described for a processor replaying all its messages is expected behavior and the idea is the processor itself is building up internal state as well as any downstream state as it processes those messages on recovery.

I am not sure what the destinations you describe might be.  Are they eventsourced processors?  Are they stateful on their own?  Meaning does the destination depend on the processor to send it data to rebuild internal state on recovery?

Generally, if you have a processor which is sending or forwarding messages to another Actor or Processor, you want to use reliable channels and confirm the messages so they don't get redelivered on recovery.  However, the downstream dependency of the processor should always be able to handle duplicate messages delivery which could happen due to network failure while confirmation message is in flight.  You may want to retain internal state in the destination such that it knows the last message id it processed and so it won't attempt to reprocess the message again.  The message delivered at the destination should have the sequence number.

Hope that helps,

Hiral Patel

[Sathish]

Hiral,

You mentioned - "Generally, if you have a processor which is sending or forwarding messages to another Actor or Processor, you want to use reliable channels and confirm the messages so they don't get redelivered on recovery."

I agree with you there. But , the default behaviour of the system is when we say extension.recover()  ,the processor replays all the messages and the reliable channel then delivers the messages that have not been acked back by the final destination.

Why does the Processor need to replay all the messages ? When we say , extension.recover(ReplayParams(1,1000)) - the Processor Is 1 replays the messages upto the sequence number of 1000.

All I really want to do is , ask the processor to send/replay only the messages that have not be acked by the final destination.

You mentioned - 

I am not sure what the destinations you describe might be.  Are they eventsourced processors?  Are they stateful on their own?  Meaning does the destination depend on the processor to send it data to rebuild internal state on recovery?

The final destinations are Actors which will post the event to a JMS Queue when they get an event - that is what they do whenever they receive an event.

Thanks for your advice , my question is , how do we make the processor not replay all the events everytime we say recover. But essentially , make the processor aware of the last successful Sequence Number and replay messages after that sequence number.

Thanks,
Sathish

[Martin Krasser]

On 19.08.13 19:46, Sathish wrote:

Hiral,

You mentioned - "Generally, if you have a processor which is sending or forwarding messages to another Actor or Processor, you want to use reliable channels and confirm the messages so they don't get redelivered on recovery."

I agree with you there. But , the default behaviour of the system is when we say extension.recover()  ,the processor replays all the messages and the reliable channel then delivers the messages that have not been acked back by the final destination.

Why does the Processor need to replay all the messages ? When we say , extension.recover(ReplayParams(1,1000)) - the Processor Is 1 replays the messages upto the sequence number of 1000.

All I really want to do is , ask the processor to send/replay only the messages that have not be acked by the final destination.

This is exactly what a reliable channel is doing (see also standalone usage of a reliable channel). You don't need a processor for that - a reliable channel is an actor that journals messages and only redelivers non-confirmed messages during recovery.

You mentioned - 

I am not sure what the destinations you describe might be.  Are they eventsourced processors?  Are they stateful on their own?  Meaning does the destination depend on the processor to send it data to rebuild internal state on recovery?

The final destinations are Actors which will post the event to a JMS Queue when they get an event - that is what they do whenever they receive an event.

... and these are actors should be used as reliable channel destinations. If you want to run them remotely, this section might help you.