Regarding Chronicle IPC without persistance

[Vachagan Balayan]

Hi, 

I'm trying to use chronicle queue for IPC between several java applications on the same machine,

but i want it not to fill up the HDD. The information passed around in this queue is not something i want to persist, 

and as i know it will be quite a lot i can predict that it will fill up the memory in a week or so.

I wonder if its possible to limit the size or number of rolled up files, for example build a chronicle that will roll up the files with some N size, and will automatically delete older ones...

My concern is not to interrupt this queue, i don't want to shut it down, clear up and reinitialize, but at the same time i don't want it to inevitably fill up the memory with the stuff i don't need to keep persisted...

Any help appreciated.

[Vachagan Balayan]

So in short i have very high frequency of messaging between processes, but i don't want to keep all the messages,

the valuable information is much smaller in volume than all the messages passed around...

[Luca Burgazzoli]

That is quite hard to reliably implement on Chronicle-Queue IMHO as only your application knows when you are done reading data and if cleanup is demanded to chronicle it may ends up removing things you have not yet consolidated.

I'd suggest to write your own logic to notify a cleanup task that you are done reading some data.

On Thu, Dec 17, 2015 at 8:11 AM, Vachagan Balayan <vachagan...@gmail.com> wrote:

So in short i have very high frequency of messaging between processes, but i don't want to keep all the messages,

the valuable information is much smaller in volume than all the messages passed around...

--
You received this message because you are subscribed to the Google Groups "Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicl...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Vachagan Balayan]

Even if i know that data is consolidated, and all my readers are perfectly catching up, i cannot remove the other files without interrupting work with that chronicle instance right?

[Luca Burgazzoli]

You can remove files while chronicle is running, obviously you need to be sure that no processes have references to such files ( i.e. on linux with lsof ) 

On Thu, Dec 17, 2015 at 10:17 AM, Vachagan Balayan <vachagan...@gmail.com> wrote:

Even if i know that data is consolidated, and all my readers are perfectly catching up, i cannot remove the other files without interrupting work with that chronicle instance right?

--

[Vachagan Balayan]

Isn't there any way to have IPC without persisting everything? 

like just a queue in memory (as i know my consumers are fast) 

and this queue gets cleared periodically until the point where the slowest consumer had reached...

Could you recommend something i could use for this? 

[Vachagan Balayan]

Thanks that seems like a solution :)

[Luca Burgazzoli]

Looks like you are looking for a multi producer/multi consumer ring buffer

--