Multiple Appenders and thread safety

[Jonathan H]

Hi,

We have been using Chronicle Queue 3.5.3 for a while very successfully. Our current usage has always been single appender to a chronicle and multiple tailers in different processes and monitoring tools. Right now I've been testing using multiple appenders (in different processes so no shared appender instance) writing to the same chronicle and our tailers break and read inconsistent data. I assume this has to do with how we have been using Chronicle Queue as in general from what I read it is safe to use with multiple appenders in different processes?

Our writers look like this:

- direct instance wrappers (e.g. orderAdd below) is generated from an interface extending Byteable

- on a write:

orderAdd.setSize(10);
/* ... */
appender.startExcerpt(orderAdd.maxSize() + 1);
appender.writeUnsignedByte(orderAddMessageType); // encoding which message type is being written
appender.write(orderAdd);
appender.finish();

Our readers poll, read and dispatch callbacks depending on message types as follows:

if(tailer.nextIndex()) {
   int messageType = tailer.readUnsignedByte();
   switch(messageType) {
   case orderAddMessageType:
         readEntry(orderAdd);
         listener.onOrderAdd(orderAdd);
         break;
   /* ... */
   }
}
/* ... */
 private <T extends Byteable> void readEntry(T t) {
   t.bytes(tailer, 1);
   tailer.finish();
 }

Like this we can write and consume a number of different message types on the same queue, but I could imagine that this handling also could lead to issues on multiple appenders? 

Thanks

Jonathan

      

 

[Rob Austin]

Jonathan 

Is your question relating to chronicle queue 3, ( if so,  note : we only answer questions on this forum relating to chronicle queue 4, is it possible for you to update to the latest version ? )

<groupId>net.openhft</groupId>

<artifactId>chronicle-queue</artifactId>

<version>4.5.13</version>

>I assume this has to do with how we have been using Chronicle Queue as in general from what I read it is safe to use with multiple appenders in different processes?

yes - in essence, the idea of chronicle-queue is to support multi appenders/tailers across multiple processes or threads. If you would like support for your current version, would you be interested in paying for a support contract ?

regards

Rob

[Jonathan H]

Thanks Rob for the quick answer, I'll evaluate chronicle queue 4 and will consider paid support.

Jonathan

[Peter Lawrey]

Hello,
  If you are using IndexedChronicle you have to ensure the appender is used in a thread safe manner. Eg by using synchronized.
   If you are using VanillaChronicle you can use this across threads or processes.

Regards, Peter.

--
You received this message because you are subscribed to the Google Groups "Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicle+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.