Kafka REST Proxy performance expectations

[Atharva Inamdar]

I'm considering using Kafka REST Proxy for producing messages only. I expect our load to be at about 4500req/s with a peak of 7200req/s.

What are the benchmarks for each instance of the proxy? How would I plan how many instances I need to deploy? 

My alternative is NGinx with a kafka module. and I know Nginx itself can handle the required load.

Thanks

Atharva

[Ewen Cheslack-Postava]

Generally you'll want to optimize throughput for the REST proxy by batching at the source, so req/s to the proxy may not be the best metric to focus on. Benchmarks obviously vary a lot by the exact parameters used (CPU, network bandwidth, batch size, message size, etc), but with appropriate parameters and enough resource headroom, the REST proxy gets throughput that is close to the native producer (assuming you've taken into account, e.g. the bandwidth overhead you need to account for).

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/ee1d4dba-e47e-4df5-b309-c1668181ff58%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen