Is Chronicle Logger Garbage free?

[EKK]

Hi,

I am testing Chronicle Logger for my application - but it seems it is not garbage free.

I have seen Log4j being now garbage free: 

https://www.infoq.com/news/2016/05/log4j-garbage-free

So I wonder if I didn't set it up properly in order to be garbage free - or if it is not GC free.

Thanks

EKK

[Luca Burgazzoli]

Chronicle Logger provides appenders for common logging framework and most of the garbage comes from such frameworks

i.e. some frameworks create an object per event which chronicle logger stores as efficent as posdible but it cannot turn off such garbage.

I'm slowly moving logger to use latest chronicle queue 4 so in the future you may combine log4j 2 with a chronicle appender.

however it should be noted that if you have other logging framework used by your dependencies you may still get some garbage, i.e. slf4j api does support a few params ony for logging methods so you may end up having an array created for each varargs call.

--
You received this message because you are subscribed to the Google Groups "Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicle+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
lb

[Peter Lawrey]

Chronicle Queue on which it is based is not entirely garbage free. It is designed to be very low garbage or much less than 1 byte per message.

Most of the garbage in Chronicle Logger comes from the API itself so we can't avoid it.
log4j2 which a rewrite of log4j has a mode of use which is low garbage however it is an asynchronous logger so data can be lost if the application crashes. Usually the last thing you did before a crash is the most important thing to log.
As such Chronicle Logger isn't asynchronous though it performs similarly.

Note: the most expensive operation you can do when logging is to include a class+line in the output. This involves taking an entire stack trace. If you do this, your choice of logger hardly matters.

Regards, Peter.

--

[Emmanuel Keskes]

HI Peter,

I am probably doing something wrong then - I can see from NetBeans Object profiler that I create a lot of ByteBufferExcerpt, byte[], AtomicBoolean, int[] and char[] (I attached a pix of the profiler).

Also here is my HelloWorld example code - 

1/ I am not sure if I should create an Excerpt for each write?

2/ should I call finish() everytime?

Thanks.

----

import com.higherfrequencytrading.chronicle.Chronicle;

import com.higherfrequencytrading.chronicle.Excerpt;

import com.higherfrequencytrading.chronicle.impl.IndexedChronicle;

import java.io.File;

import java.io.IOException;

public class ChronicleLogExample {

    private static int STRING_SIZE_OVERHEAD = 4;

    private final String tempPath = System.getProperty("user.dir");

    private final IndexedChronicle chr;

    public ChronicleLogExample() throws IOException {

        String basePrefix = tempPath + File.separator + "chronicle";

        System.out.println("base prefix: " + basePrefix);

        chr = new IndexedChronicle(basePrefix);

    }

    public static void writeToChronicle(Chronicle chr, String someString) {

        final Excerpt excerpt = chr.createExcerpt();

        excerpt.startExcerpt(someString.length() + STRING_SIZE_OVERHEAD);

        excerpt.writeBytes(someString);

        excerpt.finish();

    }

    

    public int count = 0;

    private String message = "messsage";

    private void log(){

        count++;

        writeToChronicle(chr, message);

    }

    public static void main(String[] args) throws IOException {

        ChronicleLogExample clog = new ChronicleLogExample();

        while(clog.count < 100_000_000){

            clog.log();

        }

    }

}

--
You received this message because you are subscribed to a topic in the Google Groups "Chronicle" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/java-chronicle/fEXx0JZNb20/unsubscribe.
To unsubscribe from this group and all its topics, send an email to java-chronicle+unsubscribe@googlegroups.com.

[Peter Lawrey]

I suggest you use Chronicle Queue v4 if you are going to use the queues directly. We don't provide free support for the older V3.
You can use appender.writeText(charSequence) and later dump () the queue to view it.

Peter.

[Emmanuel Keskes]

Thanks Peter - I switched to Chronicle Queue (I guess it is v4?) - it is almost GC free - and the write perf are 200ns for 99% percentile and avg = 100ns.

code below:

---

import java.io.File;

import java.io.IOException;

import net.openhft.chronicle.ChronicleQueueBuilder;

import net.openhft.chronicle.ExcerptAppender;

import net.openhft.chronicle.Chronicle;

public class ChronicleExample {

    private final Chronicle chronicle;

    private final ExcerptAppender appender;

    

    public ChronicleExample() throws IOException {

        chronicle = ChronicleQueueBuilder.vanilla(new File("my-queue2")).build();

        appender = chronicle.createAppender();

    }

    public void writeToChronicle(String msg) throws IOException {

        appender.startExcerpt();

        appender.writeChars(msg);

        appender.finish();

    }

    public static void main(String[] args) throws IOException {

        ChronicleExample clog = new ChronicleExample();

        String mess = "mess";

        int n = 1_000_000_000;

        long t1 = -System.nanoTime();

        for( int idx = 0; idx<n; idx++){

            clog.writeToChronicle(mess);

        }

        t1 += System.nanoTime();

        System.out.println("dt = " + t1 + " ns");

        double dtperelem = t1/(double)n;

        System.out.println("dt "+ dtperelem);

    }

}

[Peter Lawrey]

It's still V3.  The artifact you are looking for is chronicle-queue. Version 4.5.x

[Emmanuel Keskes]

Ok great - I have tried V4 with the code below - same perf - GC free.

thanks a lot

EKK

mport java.io.File;

import java.io.IOException;

import net.openhft.chronicle.queue.ChronicleQueueBuilder;

import net.openhft.chronicle.queue.ExcerptAppender;

import net.openhft.chronicle.queue.impl.single.SingleChronicleQueue;

public class ChronicleExample {

    

    private final ExcerptAppender appender;

    private final SingleChronicleQueue chronicle;

    

    public ChronicleExample() throws IOException {

        chronicle = ChronicleQueueBuilder.single(new File("my-queueA").getAbsolutePath()).build();

        appender = chronicle.acquireAppender();

    }

    public void writeToChronicle(String msg) throws IOException {

        appender.writeText(msg);

    }

    public static void main(String[] args) throws IOException {

        ChronicleExample clog = new ChronicleExample();

        String mess = "mess";

        int n = 1000_000_000;

[Peter Lawrey]

Note that writeText can take a recycled StringBuilder or a Bytes.

[EKK]

Hi Peter,

I have seen there is a different way to write to chronicle as described in the sandbox/replay using the BytesMarshallable interface.

https://github.com/OpenHFT/Chronicle-Queue/tree/master/chronicle-sandbox/src/main/java/net/openhft/chronicle/sandbox/replay

The BytesMarshallable way is faster than the WriteText() way - but does not seem to be garbage free.

I can see the BytesMarshallable way is not available in V4 - what would be the equivalent in V4 world?

Thanks,

EKK