Examples of Kafka Rest Producer and Consumer in JAVA

2,435 views
Skip to first unread message

Siva Santhalingam

unread,
Aug 16, 2016, 12:23:42 PM8/16/16
to Confluent Platform
Hi there,

Are there any good examples for the Producer and Consumer groups using the KAFKA REST in Java?

Thanks
Siva 

Varun Saini

unread,
Aug 16, 2016, 7:35:09 PM8/16/16
to Confluent Platform
Why do you need Producer/Consumer group, If you want to use Kafka-rest proxy? Or I am missing something?

Thanks,
Varun

Ewen Cheslack-Postava

unread,
Aug 17, 2016, 2:23:43 AM8/17/16
to Confluent Platform
Siva,

To follow up on Varun's response -- normally if you can use a native producer/consumer that would be preferable. They should universally be more efficient (as they don't require any redirection through proxies, extra encoding, etc). REST proxies for Kafka are really intended for languages that are easy to make REST requests from, but don't have good native client support.

Since Java is the language that the most recent Kafka clients are implemented in, you should use these clients unless you have good reason to use an alternative. Confluent has documentation explaining how to use these clients here: http://docs.confluent.io/3.0.0/clients/consumer.html#basic-usage with full code examples here: https://github.com/confluentinc/examples These would be more efficiently, intuitive, and preferable to the REST proxy implementation.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/e43cdf2b-5443-4859-833b-8d621536f406%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Thanks,
Ewen

Siva

unread,
Aug 17, 2016, 3:57:30 PM8/17/16
to confluent...@googlegroups.com
Thanks a lot . So the rest proxy is not suited for some low level consumer operations ?

-siva 
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/jvy2_hr4Q2I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAE1jLMM0L7W%3DO31vGgOfr2Tu71-GMJQaAbnwVFNMBR9okw5_sw%40mail.gmail.com.

Ewen Cheslack-Postava

unread,
Aug 18, 2016, 12:25:51 AM8/18/16
to Confluent Platform
Siva,

That was not what I meant to imply. There is overhead to using the REST Proxy, so when you *can* use a good, native client, that is better. But if there is not an effective client, the REST Proxy is a good choice.

In terms of low level vs high level, I think you might be referring to the old-style simple consumer vs high level consumer. The REST Proxy supports both those modes. In particular, if you want a low-level simple consumer style interface, checkout http://docs.confluent.io/3.0.0/kafka-rest/docs/api.html#get--topics-(string-topic_name)-partitions-(int-partition_id)-messages?offset=(int)[&count=(int)] which lets you consume messages from a specific topic-partition at a specified offset without requiring the coordination of a consumer group.

-Ewen

On Wed, Aug 17, 2016 at 12:57 PM, Siva <siva.san...@gmail.com> wrote:
Thanks a lot . So the rest proxy is not suited for some low level consumer operations ?

-siva 



On Aug 16, 2016, at 11:23 PM, Ewen Cheslack-Postava <ew...@confluent.io> wrote:

Siva,

To follow up on Varun's response -- normally if you can use a native producer/consumer that would be preferable. They should universally be more efficient (as they don't require any redirection through proxies, extra encoding, etc). REST proxies for Kafka are really intended for languages that are easy to make REST requests from, but don't have good native client support.

Since Java is the language that the most recent Kafka clients are implemented in, you should use these clients unless you have good reason to use an alternative. Confluent has documentation explaining how to use these clients here: http://docs.confluent.io/3.0.0/clients/consumer.html#basic-usage with full code examples here: https://github.com/confluentinc/examples These would be more efficiently, intuitive, and preferable to the REST proxy implementation.

-Ewen
On Tue, Aug 16, 2016 at 4:35 PM, Varun Saini <varunkuma...@gmail.com> wrote:
Why do you need Producer/Consumer group, If you want to use Kafka-rest proxy? Or I am missing something?

Thanks,
Varun

On Tuesday, August 16, 2016 at 11:23:42 AM UTC-5, Siva Santhalingam wrote:
Hi there,

Are there any good examples for the Producer and Consumer groups using the KAFKA REST in Java?

Thanks
Siva 

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent-platform@googlegroups.com.



--
Thanks,
Ewen

--
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/jvy2_hr4Q2I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platform+unsub...@googlegroups.com.

To post to this group, send email to confluent-platform@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/73827EB3-4F23-4387-B410-F8FE81BE01B7%40gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
Thanks,
Ewen
Reply all
Reply to author
Forward
0 new messages