Changing size of file for indexed chronicle v3.6.1.8

[Noah Taylor]

Hello,

I am writing a small number of messages and therefore would like to change the size of the file written by the appender.  Currently it defaults to 128MB and I would like smaller.  I tried change the dataBlockSize on the IndexedChronicleQueueBuilder but that doesn't seem to change the file size.  Is there a way of adjusting the file size down?

Thanks,

Noah

[Peter Lawrey]

This should be the parameter to use.  I suggest using 3.6.2 though I don't expect this will fix the issue. (or the 4.4.0 to be released by tomorrow.)

Can you show us the code you are using to build the queue?

Regards,

   Peter.

--
You received this message because you are subscribed to the Google Groups "Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicl...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Noah Taylor]

Sure.

I tried using small():

ExcerptAppender appender = ChronicleQueueBuilder.indexed(fileInPath).small().build()

            .createAppender();

or just setting the datablock directly:

ExcerptAppender  appender = ChronicleQueueBuilder.indexed(fileInPath).dataBlockSize(1024*2).build()

            .createAppender();

Neither seem to affect the file size on disk.  I tried upgrading to 3.6.2 but that didn't work either.

[Peter Lawrey]

I have added this test.

    @Test

    public void testFileSizes() throws IOException {

        final String basePath = getTestPath();

        try (final Chronicle chronicle = ChronicleQueueBuilder.indexed(basePath)

                .dataBlockSize(256 << 10)

                .indexBlockSize(64 << 10)

                .build()) {

            ExcerptAppender appender = chronicle.createAppender();

            int objects = 100;

            for (int i = 0; i < objects; i++) {

                appender.startExcerpt();

                appender.writeObject(BigDecimal.valueOf(i % 1000));

                appender.finish();

            }

            appender.close();

            Assert.assertEquals(256.0, new File(basePath + ".data").length() / 1024.0, 0.0);

            Assert.assertEquals(64.0, new File(basePath + ".index").length() / 1024.0, 0.0);

        } finally {

            assertClean(basePath);

        }

    }

Can you try it?  

Note this will not shrink an existing file.  The only way to do that is to copy the contents to a smaller queue and replace the original.

Regards,

    Peter.

[Noah Taylor]

Thanks for the test case.  I used your case and it passed.

The issue was I did not have the try catch block in my code, once I added that it worked.  I'm a bit surprised and while I think it is good practice to use a try catch on a closeable, how come it is necessary in this case for sizing the file?

Thanks for your help!

[Noah Taylor]

Ok so I think I've solved the issue.  The code I have generates a file and then subsequently reads from it.  When the file is initially written the size is as specified by the appender.  The issue occurs when I create a new tailer on that file.  The tailer needs to be set to have the same block size otherwise the file will be expanded back to the default block size of 128MB.  Dohhh!

[Peter Lawrey]

Chronicle Queue v4 stores the configuration in the start of the file in a readable form so you can see this more clearly.