Compression with Kafka REST

[Derar Alassi]

Hi guys,

I have Kafka REST 1.0.1 installed. It works fine but I the network traffic is huge compared to when compression is enabled. Is it possible to compress the HTTP request on the client side before invoking the Kafka REST service?

Thanks,

Derar

[Ewen Cheslack-Postava]

Hi Derar,

This hasn't been implemented yet, but we're tracking the issue here: https://github.com/confluentinc/rest-utils/issues/18 If we add support in rest-utils, we'll get support across multiple projects. I suspect the implementation should be a relatively small patch since it should be able to take advantage of the existing support in Jetty.

However, keep in mind that since gzip is the most broadly supported compression for HTTP requests, you'll also pay the CPU cost for it. Compared to snappy, which is very fast, gzip can have pretty high overhead.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/5e8f748b-6176-4df9-a203-4016ef0ef7b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen

[Derar Alassi]

Thanks Ewen. Chose gzip because it's supported by HTTP web servers. But I agree, there will be CPU cost that the client and the server will have to pay for.

On Thursday, October 29, 2015 at 2:47:19 PM UTC-7, Ewen Cheslack-Postava wrote:

Hi Derar,

This hasn't been implemented yet, but we're tracking the issue here: https://github.com/confluentinc/rest-utils/issues/18 If we add support in rest-utils, we'll get support across multiple projects. I suspect the implementation should be a relatively small patch since it should be able to take advantage of the existing support in Jetty.

However, keep in mind that since gzip is the most broadly supported compression for HTTP requests, you'll also pay the CPU cost for it. Compared to snappy, which is very fast, gzip can have pretty high overhead.

-Ewen

On Fri, Oct 23, 2015 at 5:16 PM, Derar Alassi <derar....@gmail.com> wrote:

Hi guys,

I have Kafka REST 1.0.1 installed. It works fine but I the network traffic is huge compared to when compression is enabled. Is it possible to compress the HTTP request on the client side before invoking the Kafka REST service?

Thanks,

Derar

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/5e8f748b-6176-4df9-a203-4016ef0ef7b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen