Kafka streams batch processing topology

40 views
Skip to first unread message

Ra Sa

unread,
Feb 1, 2021, 3:08:15 PM2/1/21
to Confluent Platform
Hello,

I am trying to find out what are the best options and approaches to process events by batches. My use case: I my consumers to wake up every 5 mins or so and consumes say 100 messages at a time so that I can process them all at once and send them to an external API all 100 messages at once - this is required because this external API does not handle too many requests at a time. I've been researching options and so far I find out 1. using poll interval and poll records. 2. use windowing. Any thoughts?

Thanks,
RS
Reply all
Reply to author
Forward
0 new messages