Recommended number of concurrent writer threads

71 views
Skip to first unread message

jaisins

unread,
Jan 15, 2016, 4:24:48 PM1/15/16
to Chronicle
My use case requires writing several thousand messages per sec. 

I use a threadpool, where each thread writes to CQ, in response a prior step. There are about 20-100 threads. 
In order to make things threadsafe, each thread does a createappender(), writes, calls finish() and then close(). 
This overhead consumes most of the resources, ends up with thousands of files (presumably each thread creates a new file on every write operation). The time to write to queue increases from 2- 2000 ms within a few seconds. It is nowhere near ns rates that I assume are normal.
What is the safe number of threads for concurrent writes to ChronicleQueue to achieve a ns rate.
Is my use case, not suitable for ChronicleQueue.


Vachagan Balayan

unread,
Feb 9, 2016, 12:05:49 PM2/9/16
to Chronicle
Pardon me, but as i understand you're in the same java process, wouldn't it be more efficient if you used submit all writable stuff to a single thread pool that would write it all to queue? several thousand is not that much as i know...

Anyway as far as i understand you want to have only one writer per process per queue... if you have multiple writer threads you better synchronize inside your java app... 
Reply all
Reply to author
Forward
0 new messages