How to speed up data imports from ring buffers in a java client?

15 views
Skip to first unread message

JDev

unread,
Jul 26, 2023, 6:30:12 AM7/26/23
to Hazelcast
We have a third party application producing data at a very high speed and storing it in multiple ring buffers (hazelcast cluster). 

The consumer application (another third party app) uses this java hazelcast library to read ring buffer data. It calls final byte[] item = ringbuffer.readOne(sequence) api for reads. This turns out to be very slow. The consumer takes more than a day to read all data during peak loads.

What are general strategies to concurrently read data from hazelcast cluster? Are there any java references that I can look into?

Thanks.

oz...@hazelcast.com

unread,
Aug 8, 2023, 4:09:43 AM8/8/23
to Hazelcast
RingBuffer is hosted on a single partition & single member so you can't expect the same performance of a map that is distributed on all members. 
One suggestion would be storing your data in a map and consuming via Jet jobs: https://docs.hazelcast.com/hazelcast/5.3/pipelines/stream-imap 
Reply all
Reply to author
Forward
0 new messages