ChronicleQueue disk usage

Alper Tekinalp

unread,

Dec 22, 2015, 12:20:53 PM12/22/15

to Chronicle

Hi.

I have a ChronicleQueue as:

Chronicle priorityChronicle = ChronicleQueueBuilder. vanilla(chronicleDest). cycleLength(60 * 60 * 1000). cycleFormat(CYCLE_FORMAT). build();

public void enqueue(String event) { 
    synchronized (appenderLock) { 
        appender.startExcerpt(); 
        appender.writeEnum(event); 
        appender.finish(); 
    }
 }

When I start 2 Thread that just enqueue a dummy string to queue, proccess creates roughly 40GB files in a few (1 - 2) minutes which is total empty space in my pc. When I open a file I can see there are only a few lines that consist data and the other part of the file contains only 0000 0000 like lines.

e1ff ffff 1d54 6869 7320 6973 2061 2064
756d 6d79 2073 7472 696e 6720 6f62 6a65
6374 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
....

1 - Is it normal that many files created? Bug or feature? :)
2 - Why new files created when the others not filled?
3 - Is there a size limit? I mean could you say ChronicleQueue need 100 GB empty space which it will use no matter if other files filled with data or not?

My dummy program is in the attachment.

Regards.

PersistentQueueTest.zip

Alper Tekinalp

unread,

Dec 29, 2015, 10:16:46 AM12/29/15

to Chronicle

Hi.

Anyone has an idea?

22 Aralık 2015 Salı 19:20:53 UTC+2 tarihinde Alper Tekinalp yazdı:

Peter Lawrey

unread,

Dec 29, 2015, 2:23:40 PM12/29/15

to java-ch...@googlegroups.com

Hello Alper,

Can you provide a simple example which reproduces this problem?

How many threads are you creating? Each thread has it's own file for concurrency and need to have it's own appender.

If you don't have a limited number of threads, I suggest you use an indexed queue with a lock as you have done.

Regards,

Peter.

--
You received this message because you are subscribed to the Google Groups "Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicl...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Luca Burgazzoli

unread,

Dec 30, 2015, 1:13:04 AM12/30/15

to java-ch...@googlegroups.com

You should not share the same appenders/tailers among threads, each one need it's own instance.

--
lb

Reply all

Reply to author

Forward