When is StoreFileListener called?

[Mickael Marrache]

Hi,

We have two applications A and B. A is a writer, B is a reader. When A is started, A creates a new file (hourly based) and starts writing to it. When B is started, it detects the file and starts reading from it.

B also has a listener that archive the file when processing is done (StoreFileListener). When the hour ends, we sometimes see that B archive the file (simple move operation) while A continues writing to it. This is possible because file move is a simple file rename, therefore A is able to continue writing to it. Therefore, I wonder what triggers the StoreFileListener to be called. I can think of two possibilities:

1. Discovery of next file created by A

2. Time based decision (therefore not always in sync with A)

Can you please confirm how the listener call is triggered?

We are using Chronicle Queue 4.5.12.

Thanks,

Mickael

[Peter Lawrey]

Hi Michael,

The caller is triggered when the node is ready to write/use a new file. If you have a node which is only reading, this might make as much sense. 

The best way for the reader to determine when the old files is not longer needed is when the reader changes the cycle in its index. Until you have read the end of file marker, you don't know if more messages might be written.

Regards,

   Peter.

--
You received this message because you are subscribed to the Google Groups "Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicle+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mickael Marrache]

Hi Peter,

Are you saying that I can't rely on the StoreFileListener in my case? If not, can you please provide a code snippet describing how to poll the file for messages until EOF is reached?

Thanks!

[Peter Lawrey]

You are already polling to read.

You need to check if RollCycle.toCycle (tailer.index ()) changes.

[Mickael Marrache]

I use tailer.cycle() to retrieve the current cycle. Is it equivalent to RollCycle.toCycle (tailer.index ())?

Also, how can I get the file name from the cycle in order to move the file after processing?

Last question, when only one tailer reads from the queue, does a cycle change mean that the file is not written anymore by appenders?

Thanks!

[Peter Lawrey]

Hello Mickael,

The index is composed from the cycle () so you are right.

A tailer will only change cycle when an end of file marker has been added to the older cycle.

I thought the StoreFileListener had the file name but it's been a while.

[Mickael Marrache]

I can't use StoreFileListener because of the issue I mentioned in my question - when onRelease is called, the appender may still be writing to the previous cycle.

Therefore, you suggested to poll the cycle after every read and detect when it changes.

Therefore I don't have the name of the file associated with the previous cycle - I need to compute it.

[Peter Lawrey]

The listener tells you about files as they are created. Ie the information is thete, it doesn't make decisions about when you could remove them.

[Mickael Marrache]

I understand - therefore how can I get the file name from the cycle?

[Rob Austin]

I add this to the readme recently, see 

How to find the current cq4 chroncile-queue is writing to

WireStore wireStore = queue.storeForCycle(queue.cycle(), 0, false);
System.out.println(wireStore.file().getAbsolutePath());

To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicl...@googlegroups.com.

[Mickael Marrache]

So I need to create a WireStore for that? Event if the tailer is reading from a new cycle?