Examples of Kafka Rest Producer and Consumer in JAVA

Siva Santhalingam

unread,

Aug 16, 2016, 12:23:42 PM8/16/16

to Confluent Platform

Hi there,

Are there any good examples for the Producer and Consumer groups using the KAFKA REST in Java?

Thanks

Siva

Varun Saini

unread,

Aug 16, 2016, 7:35:09 PM8/16/16

to Confluent Platform

Why do you need Producer/Consumer group, If you want to use Kafka-rest proxy? Or I am missing something?

Thanks,

Varun

Ewen Cheslack-Postava

unread,

Aug 17, 2016, 2:23:43 AM8/17/16

to Confluent Platform

Siva,

To follow up on Varun's response -- normally if you can use a native producer/consumer that would be preferable. They should universally be more efficient (as they don't require any redirection through proxies, extra encoding, etc). REST proxies for Kafka are really intended for languages that are easy to make REST requests from, but don't have good native client support.

Since Java is the language that the most recent Kafka clients are implemented in, you should use these clients unless you have good reason to use an alternative. Confluent has documentation explaining how to use these clients here: http://docs.confluent.io/3.0.0/clients/consumer.html#basic-usage with full code examples here: https://github.com/confluentinc/examples These would be more efficiently, intuitive, and preferable to the REST proxy implementation.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/e43cdf2b-5443-4859-833b-8d621536f406%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen

Siva

unread,

Aug 17, 2016, 3:57:30 PM8/17/16

to confluent...@googlegroups.com

Thanks a lot . So the rest proxy is not suited for some low level consumer operations ?

-siva

You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/jvy2_hr4Q2I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAE1jLMM0L7W%3DO31vGgOfr2Tu71-GMJQaAbnwVFNMBR9okw5_sw%40mail.gmail.com.

Ewen Cheslack-Postava

unread,

Aug 18, 2016, 12:25:51 AM8/18/16

to Confluent Platform

Siva,

That was not what I meant to imply. There is overhead to using the REST Proxy, so when you *can* use a good, native client, that is better. But if there is not an effective client, the REST Proxy is a good choice.

In terms of low level vs high level, I think you might be referring to the old-style simple consumer vs high level consumer. The REST Proxy supports both those modes. In particular, if you want a low-level simple consumer style interface, checkout http://docs.confluent.io/3.0.0/kafka-rest/docs/api.html#get--topics-(string-topic_name)-partitions-(int-partition_id)-messages?offset=(int)[&count=(int)] which lets you consume messages from a specific topic-partition at a specified offset without requiring the coordination of a consumer group.

-Ewen

On Wed, Aug 17, 2016 at 12:57 PM, Siva <siva.san...@gmail.com> wrote:

Thanks a lot . So the rest proxy is not suited for some low level consumer operations ?

-siva

On Aug 16, 2016, at 11:23 PM, Ewen Cheslack-Postava <ew...@confluent.io> wrote:

Siva,

To follow up on Varun's response -- normally if you can use a native producer/consumer that would be preferable. They should universally be more efficient (as they don't require any redirection through proxies, extra encoding, etc). REST proxies for Kafka are really intended for languages that are easy to make REST requests from, but don't have good native client support.

Since Java is the language that the most recent Kafka clients are implemented in, you should use these clients unless you have good reason to use an alternative. Confluent has documentation explaining how to use these clients here: http://docs.confluent.io/3.0.0/clients/consumer.html#basic-usage with full code examples here: https://github.com/confluentinc/examples These would be more efficiently, intuitive, and preferable to the REST proxy implementation.

-Ewen

On Tue, Aug 16, 2016 at 4:35 PM, Varun Saini <varunkuma...@gmail.com> wrote:

Why do you need Producer/Consumer group, If you want to use Kafka-rest proxy? Or I am missing something?

Thanks,
Varun

On Tuesday, August 16, 2016 at 11:23:42 AM UTC-5, Siva Santhalingam wrote:
Hi there,

Are there any good examples for the Producer and Consumer groups using the KAFKA REST in Java?

Thanks
Siva

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/e43cdf2b-5443-4859-833b-8d621536f406%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Thanks,
Ewen

--

You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/jvy2_hr4Q2I/unsubscribe.

To unsubscribe from this group and all its topics, send an email to confluent-platform+unsub...@googlegroups.com.

To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAE1jLMM0L7W%3DO31vGgOfr2Tu71-GMJQaAbnwVFNMBR9okw5_sw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/73827EB3-4F23-4387-B410-F8FE81BE01B7%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen

Reply all

Reply to author

Forward