Using a Chronicle as an event log for replay in the event of a crash

[Bill Poitras]

I'm looking to use Chronicle to create an event log for my app, so when if the application crashes, I can recreate application state up until the crash, and then continue processing normally.  

What is the recommended way to keep track of the last processed excerpt?

--bill

[Peter Lawrey]

Chronicle finds the last valid entry (lazily) from the index file when you open it. This determines the size() and where toEnd() jumps to.

--
You received this message because you are subscribed to the Google Groups "Java Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicl...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Bill Poitras]

Thanks for the quick reply.

I understand that it's easy to find the end to append to after a restart.

I want one thread to write to a chronicle while another processes the data. The first may get ahead of the second because the second writes the results to an output JMS queue. So after restart I need to know not only the last entry but the last processed entry

Perhaps a second chronicle where I write the last processed index from the first? Like a cursor.

--bill

[Peter Lawrey]

I would have a chronicle log the details of every JMS message sent as the source index and timings. I used nanoTime but current time millis might be fine. By looking at the last entry, you know where to read from.

Btw if you can batch messages it can be more efficient. You can read messages until you run out or reach a size and then commit.

[Bill Poitras]

Thanks!

--bill

[Bill Poitras]

I've used your suggestion for logging the input, and am also planning on doing this for the output as well.  A log of output messages, with a separate chronicle for recording which output events have been sent.  This allows my processing thread to only care about writing to the Output chronicle.

For batching, would you suggest I submit the entire output in a batch the same size as the input:

- Read input batch

- Begin excerpt in the output log

- Record output for each event in the the batch

- Finish output excerpt once the last input event is processed.

--bill

[Peter Lawrey]

For chronicle 1.x you will get a performance improvement batching up to excerpts of 200 bytes.
For chronicle 2.x you will get an improvement if you batch messages smaller than 50 bytes.
If you have batches over 4 KB you could consider making them smaller.
Anything between 200 and 4000 bytes doesn't matter too much.