Post large amount of data to kafka-rest

[Stephane Drouin]

Hi,

I haven been unsuccessful in trying to send large amount of data to kafka-rest interface via POST .

By large I mean 20-30 MB.

The post command always return 400.

Seems like my content type is either the wrong one of perhaps this is unsupported?

Any example would be appreciated.

Thanks

 

[Ewen Cheslack-Postava]

I haven't tried sending data that large before. First, are you configuring the REST proxy with the necessary client configs (e.g. max.request.size) to allow large messages? The default max message size is much smaller than 20MB. You'll also need to make sure you're topics are configured to accept messages that large.

If that is all correctly configure, my next guess is that you're running into Jetty's default limit on request size. I only did a quick search so I'm not certain this is the correct config to be looking at, but  https://www.eclipse.org/jetty/documentation/9.2.7.v20150116/configuring-form-size.html suggests that the limit is 200K (I'm only unsure because this actually isn't a form, it's just the request entity size).

Unfortunately, if that's the issue (which you can probably find if you turn up the log level or maybe turn on debug=true to get the error message returned in the response), you'd need to adjust the Jetty config and that will require recompiling the REST proxy at the moment.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/a0c6ef08-6671-487f-aa75-5ad2fccbd633%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen