Kafka Stream Java Example Application stuck

515 views

Skip to first unread message

Lim Kim Tee

unread,

Apr 6, 2017, 2:10:22 AM4/6/17

to Confluent Platform

Hi all,

I have an issue with regards to Kafka. I have posted the question here.

I have re-run the example and still couldnt see the System.out.println that I have put it in the code.

I have tried the example in the Ubuntu 16.04 but still it didnt work.

Please advise what should I do. Thanks.

Lim Kim Tee

unread,

Apr 6, 2017, 2:15:04 AM4/6/17

to Confluent Platform

Just ao add on.

I deploy confluent platform community edition 3.2.0 in Ubuntu 16.04.

Lim Kim Tee

unread,

Apr 6, 2017, 2:24:49 AM4/6/17

to Confluent Platform

Hi I just want to clarify that the examples I have re-run is using Eclipse in Ubuntu 16.04 desktop.

Following the link I provided from earlier, I have re-run in Eclipse on Windows.

I have tried to run the examples in Ubuntu server 16.04, the OS i have deployed Confluent platform. It works.

Just somehow I am unable to run the example using both of my WIndows and Ubuntu Desktop.

I would like to ask if there is other thing I might have missed out? Thanks.

Matthias J. Sax

unread,

Apr 7, 2017, 1:01:21 PM4/7/17

to confluent...@googlegroups.com

Do you deploy CP3.2 locally for each setup? Or do you have one CP
deployment and just run Kafka Streams app in different environments?

-Matthias

> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to confluent-platf...@googlegroups.com
> <mailto:confluent-platf...@googlegroups.com>.
> To post to this group, send email to confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

signature.asc

Lim Kim Tee

unread,

Apr 12, 2017, 10:11:30 PM4/12/17

to confluent...@googlegroups.com

Hi Matthias,

Sorry for the late reply as I just returned from an holiday.

I have deployed CP 3.2 on a single ubuntu 16.04 setup. So all the packages from CP 3.2 are all in a single Ubuntu 16.04 since I wanted to test out how Kafka works.

My setup consists of the following:

1, 1 x CP 3.2 in Ubuntu Server 16.04

2. 1 x Windows 10 Eclipse Mars Java 1.8 to test out the Kafka Java API, I have also tried using Ubuntu Desktop 16.04 Eclipse to see is it an OS issue. So far it is the same.

Please advise. Thanks.

Regards,

Kim Tee

On Sat, Apr 8, 2017 at 1:01 AM, Matthias J. Sax <matt...@confluent.io> wrote:

Do you deploy CP3.2 locally for each setup? Or do you have one CP
deployment and just run Kafka Streams app in different environments?

-Matthias

On 4/5/17 11:24 PM, Lim Kim Tee wrote:
> Hi I just want to clarify that the examples I have re-run is using
> Eclipse in Ubuntu 16.04 desktop.
> Following the link I provided from earlier, I have re-run in Eclipse on
> Windows.
>
> I have tried to run the examples in Ubuntu server 16.04, the OS i have
> deployed Confluent platform. It works.
> Just somehow I am unable to run the example using both of my WIndows and
> Ubuntu Desktop.
>
> I would like to ask if there is other thing I might have missed out? Thanks.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to confluent-platform+unsub...@googlegroups.com
> <mailto:confluent-platform+unsub...@googlegroups.com>.
> To post to this group, send email to confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io.

Matthias J. Sax

unread,

Apr 13, 2017, 11:44:51 AM4/13/17

to confluent...@googlegroups.com

Thanks for the update.

It's hard to tell...

- can you please try out bin/kafka-console-consumer script to see if
you are able to connect to the brokers in the first place
- do you see anything suspicious in the Kafka Streams logs?
- please also check, if the Streams app shows app as a consumer group
- you can use bin/kafka-consumer-groups script (Streams uses it's
application ID as group.id internally)

Hope that this reveals what the problem might be.

-Matthias

> > an email to confluent-platf...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>
> > <mailto:confluent-platf...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>>.
> > To post to this group, send email to confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>
> > <mailto:confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>>.

> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com>
> >

> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com?utm_medium=email&utm_source=footer>>.
> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to a topic in
> the Google Groups "Confluent Platform" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe

> <https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email to

> confluent-platf...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>.

> To post to this group, send email to

> confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io
> <https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io>.

> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to confluent-platf...@googlegroups.com
> <mailto:confluent-platf...@googlegroups.com>.
> To post to this group, send email to confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com
> <https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

signature.asc

Lim Kim Tee

unread,

Apr 16, 2017, 10:49:14 PM4/16/17

to confluent...@googlegroups.com

Hi Matthias,

I have no problem connecting the broker at the server hosting the confluent platform using the following command:

/usr/bin/kafka-console-consumer --zookeeper localhost:2181 --topic streams-wordcount-output --from-beginning --formatter kafka.tools.DefaultMessageFormatter --property print.key=true --property key.deserializer=org.apache.kafka.common.serialization.StringDeserializer --property value.deserializer=org.apache.kafka.common.serialization.LongDeserializer

however, when I tried to connect remotely to the broker using my Ubuntu Desktop with the following command:

/usr/bin/kafka-console-consumer --zookeeper EXTERNAL_IP:2181 --topic streams-wordcount-output --from-beginning --formatter kafka.tools.DefaultMessageFormatter --property print.key=true --property key.deserializer=org.apache.kafka.common.serialization.StringDeserializer --property value.deserializer=org.apache.kafka.common.serialization.LongDeserializer

i have the following error:

[2017-04-17 10:29:40,858] WARN Fetching topic metadata with correlation id 7 for topics [Set(streams-wordcount-output)] from broker [BrokerEndPoint(0,127.0.0.1,9092)] failed (kafka.client.ClientUtils$)

java.nio.channels.ClosedChannelException

at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)

at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)

at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)

at kafka.producer.SyncProducer.send(SyncProducer.scala:124)

at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:60)

at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:95)

at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:68)

at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)

[2017-04-17 10:29:40,858] WARN [console-consumer-65417_imerino-virtual-machine-1492396179036-d8da2b51-leader-finder-thread], Failed to find leader for Set(streams-wordcount-output-0) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread)

kafka.common.KafkaException: fetching topic metadata for topics [Set(streams-wordcount-output)] from broker [ArrayBuffer(BrokerEndPoint(0,127.0.0.1,9092))] failed

at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:74)

at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:95)

at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:68)

at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)

Caused by: java.nio.channels.ClosedChannelException

at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)

at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)

at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)

at kafka.producer.SyncProducer.send(SyncProducer.scala:124)

at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:60)

I made changes to the following command

/usr/bin/kafka-console-consumer --bootstrap-server EXTERNAL_IP:9020 --topic streams-wordcount-output --from-beginning --formatter kafka.tools.DefaultMessageFormatter --property print.key=true --property key.deserializer=org.apache.kafka.common.serialization.StringDeserializer --property value.deserializer=org.apache.kafka.common.serialization.LongDeserializer

The command do run but it is stuck there.

I just run a /usr/bin/kafka-consumer-groups -zookeeper EXTERNAL_IP:2181 --list

this is the output.

Note: This will only show information about consumers that use ZooKeeper (not those using the Java consumer API).

Consumer-1491459781490-avro

Consumer-1491297810627-json

Consumer-1491459783592-json

Consumer-1491274811868-json

Consumer-1491207198673-avro

Consumer-1491458584571-avro

Consumer-1491200773537-json

Consumer-1490930463703-avro

Consumer-1491459781400-binary

Consumer-1491274842650-avro

Consumer-1491290159483-avro

Consumer-1490943639569-json

Consumer-1491198750272-avro

Consumer-1491297786680-json

Consumer-1490943325907-avro

Consumer-1491201155890-json

Consumer-1490943007087-avro

Consumer-1491206361865-avro

Consumer-1491290159528-json

Consumer-1491273474089-avro

Consumer-1491290673106-json

Consumer-1491200517614-avro

Consumer-1492392272041-binary

Consumer-1491297810619-binary

Consumer-1491206357419-json

Consumer-1491274773029-json

Consumer-1491297807222-avro

Consumer-1491289975010-json

Consumer-1491293853165-binary

Consumer-1491201154118-json

Consumer-1491294098918-binary

Consumer-1491290697345-json

Consumer-1491201244375-json

Consumer-1491206642723-avro

Consumer-1491206365882-avro

Consumer-1491206235381-json

Consumer-1491459783967-binary

Consumer-1490943325937-json

Consumer-1491293844375-json

Consumer-1491297810454-json

Consumer-1491200818783-json

Consumer-1491201085342-json

Consumer-1492390447538-avro

Consumer-1491200512879-json

Consumer-1491188848240-json

Consumer-1491200664219-json

Consumer-1491202063135-json

Consumer-1491297785256-json

Consumer-1491459783903-json

Consumer-1491297810811-binary

Consumer-1491289947339-avro

Consumer-1491206238590-json

Consumer-1491206240661-json

Consumer-1491200592777-json

Consumer-1490943663514-avro

Consumer-1491293831344-json

Consumer-1491201004334-json

Consumer-1491294098794-json

Consumer-1490943008744-avro

Consumer-1490943615825-json

Consumer-1491200711860-json

Consumer-1491297809544-avro

Consumer-1491297798910-avro

Consumer-1491290691810-json

Consumer-1491290695973-avro

Consumer-1491294097308-avro

Consumer-1492390439292-avro

Consumer-1492392271908-json

Consumer-1491293851593-avro

Consumer-1491274840588-json

Consumer-1490943177015-avro

Consumer-1490943639536-avro

Consumer-1491274773001-avro

Consumer-1490943663551-json

Consumer-1490943177043-json

Consumer-1491290659795-avro

Consumer-1491273474116-json

Consumer-1492392270894-avro

Consumer-1491293830023-avro

Consumer-1491297785223-avro

Consumer-1491207200671-avro

Consumer-1491294091414-json

Consumer-1491459780210-avro

Consumer-1491459782701-avro

Consumer-1491293853022-json

Consumer-1491200769460-json

Consumer-1491200682083-json

Consumer-1491200714343-json

Consumer-1491359473743-avro

Consumer-1490861861402-json

Consumer-1491273495777-avro

Consumer-1491297786818-binary

Consumer-1491201303616-json

Consumer-1490943333808-avro

Consumer-1490943615794-avro

Consumer-1491200775365-json

Consumer-1491200996736-json

Consumer-1491290659823-json

Consumer-1491198743588-json

Consumer-1491289947367-json

Consumer-1491459778848-avro

Consumer-1491201000622-json

Consumer-1491294078423-json

Consumer-1490942848375-avro

Consumer-1490930432556-avro

Consumer-1491293830050-json

Consumer-1491201002605-json

Consumer-1491201247012-json

Consumer-1491458581300-json

Consumer-1491289977567-avro

Consumer-1491200771989-json

Consumer-1491273495805-json

console-consumer-69259

Consumer-1491459784058-binary

Consumer-1490943333838-json

Consumer-1490930429433-json

Consumer-1491459780117-json

Since I am running in a console, "console-consumer-69259" seem to be a likely group. I have executed this command

/usr/bin/kafka-consumer-groups -zookeeper 192.168.4.156:2181 --describe --group console-consumer-69259

this is the output

Note: This will only show information about consumers that use ZooKeeper (not those using the Java consumer API).

TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID

It seem like there is no output.

I cant find the log for kafka stream.

Please advise if I am doing correctly. Sorry for the lengthy post. Thanks.

regards,

Kim

> > an email to confluent-platform+unsub...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>
> > <mailto:confluent-platform+unsub...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>>.
> > To post to this group, send email to confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>
> > <mailto:confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>>.

> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com>
> >

> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com?utm_medium=email&utm_source=footer>>.
> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to a topic in
> the Google Groups "Confluent Platform" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe
> <https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email to

> confluent-platform+unsub...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>.

> To post to this group, send email to

> confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io
> <https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to confluent-platform+unsub...@googlegroups.com
> <mailto:confluent-platform+unsub...@googlegroups.com>.
> To post to this group, send email to confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com
> <https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/2f5ef71d-df58-e5a3-aaa8-e5d68a4c0f7d%40confluent.io.

Lim Kim Tee

unread,

Apr 16, 2017, 10:58:30 PM4/16/17

to confluent...@googlegroups.com

Hi Matthias,

Please take note that my port number for bootstrap-server is 9092. I made an error when i type the command

/usr/bin/kafka-console-consumer --bootstrap-server EXTERNAL_IP:9092

regards,

Kim Tee

> > an email to confluent-platform+unsubscribe@googlegroups.com
> <mailto:confluent-platform%2Bunsub...@googlegroups.com>
> > <mailto:confluent-platform+unsubsc...@googlegroups.com
> <mailto:confluent-platform%2Bunsub...@googlegroups.com>>.

> > To post to this group, send email to confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>
> > <mailto:confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>>.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com>
> >
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/confluent-platform/f11e76ff-1a4a-4c15-9c13-54b49060e5ce%40googlegroups.com?utm_medium=email&utm_source=footer>>.
> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to a topic in
> the Google Groups "Confluent Platform" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe
> <https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email to

> confluent-platform+unsubscribe...@googlegroups.com
> <mailto:confluent-platform%2Bunsub...@googlegroups.com>.

> To post to this group, send email to
> confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io
> <https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to confluent-platform+unsubscribe@googlegroups.com
> <mailto:confluent-platform+unsubsc...@googlegroups.com>.

> To post to this group, send email to confluent-platform@googlegroups.com
> <mailto:confluent-platform@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com
> <https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent-platform@googlegroups.com.

Lim Kim Tee

unread,

Apr 17, 2017, 10:09:06 PM4/17/17

to confluent...@googlegroups.com

Hi Matthias,

After some investigation, I realized that I need to include the following setting in the server.properties

host.name=IP_ADDRESS_ASSIGNED_TO_THIS_MACHINE

then i am able to receive it using the /usr/bin/kafka-console-consumer --bootstrap-server EXTERNAL_IP:9092

however, when I try to run the code https://github.com/apache/kafka/blob/0.10.2/streams/examples/src/main/java/org/apache/kafka/streams/examples/wordcount/WordCountDemo.java in eclipse, it does not work.

Please advise. Thanks.

Regards,

Kim Tee

Lim Kim Tee

unread,

Apr 17, 2017, 10:44:34 PM4/17/17

to confluent...@googlegroups.com

Hi Matthias,

After I re-create the Kafka topic for the tutorial, I was able to re-run the java example of WordCountDemos using eclipse with the IP change from localhost to EXTERNAL_IP.

Regards,

Kim Tee

Lim Kim Tee

unread,

Apr 17, 2017, 10:59:09 PM4/17/17

to confluent...@googlegroups.com

Hi Matthias,

After looking through the code, just realize that I need to run a reset everytime i need to run the demo. My case has been solved.

Just that i find it odd that by setting host.name to the IP address assigned to the machine, I am able to run remotely and I am unable to run it on localhost. If I comment host.name, I am able to run it on localhost but not from remote address even though the documentation mentioned that not setting it, Kafka will bind to all interfaces. Reference: https://kafka.apache.org/090/documentation.html#brokerconfigs. Any idea what should I do to make it run on both remote and localhost?

Thanks.

Regards,

Kim Tee

Matthias J. Sax

unread,

Apr 17, 2017, 11:22:03 PM4/17/17

to confluent...@googlegroups.com

Glad it works (at least somehow)!

I am not sure atm why setting `host.name` changes the behavior. However,
you link is not correct. Using CP 3.2 you need to consider docs for
Kafka 0.10.2

And for 0.10.2 broker setting `host.name` is actually deprecated.

I would also recommend to use the Confluent docs:
http://docs.confluent.io/current/

-Matthias

On 4/17/17 7:59 PM, Lim Kim Tee wrote:
> Hi Matthias,
>

> After looking through the code, just realize that I need to run a reset
> everytime i need to run the demo. My case has been solved.

> Just that i find it odd that by setting host.name <http://host.name> to

> the IP address assigned to the machine, I am able to run remotely and I
> am unable to run it on localhost. If I comment host.name

> <http://host.name>, I am able to run it on localhost but not from remote

> address even though the documentation mentioned that not setting it,
> Kafka will bind to all interfaces. Reference:
> https://kafka.apache.org/090/documentation.html#brokerconfigs. Any idea
> what should I do to make it run on both remote and localhost?
>
> Thanks.
>
> Regards,
> Kim Tee
>
> On Tue, Apr 18, 2017 at 10:44 AM, Lim Kim Tee <stealth...@gmail.com
> <mailto:stealth...@gmail.com>> wrote:
>
> Hi Matthias,
>
> After I re-create the Kafka topic for the tutorial, I was able to
> re-run the java example of WordCountDemos using eclipse with the IP
> change from localhost to EXTERNAL_IP.
>
> Regards,
> Kim Tee
>
> On Tue, Apr 18, 2017 at 10:09 AM, Lim Kim Tee
> <stealth...@gmail.com <mailto:stealth...@gmail.com>> wrote:
>
> Hi Matthias,
>
> After some investigation, I realized that I need to include the
> following setting in the server.properties
>

> host.name <http://host.name>=IP_ADDRESS_ASSIGNED_TO_THIS_MACHINE

>
> then i am able to receive it using
> the /usr/bin/kafka-console-consumer --bootstrap-server
> EXTERNAL_IP:9092
>
> however, when I try to run the
> code https://github.com/apache/kafka/blob/0.10.2/streams/examples/src/main/java/org/apache/kafka/streams/examples/wordcount/WordCountDemo.java
> <https://github.com/apache/kafka/blob/0.10.2/streams/examples/src/main/java/org/apache/kafka/streams/examples/wordcount/WordCountDemo.java>
> in eclipse, it does not work.
> Please advise. Thanks.
>
> Regards,
> Kim Tee
>
> On Mon, Apr 17, 2017 at 10:58 AM, Lim Kim Tee
> <stealth...@gmail.com <mailto:stealth...@gmail.com>> wrote:
>
> Hi Matthias,
>
> Please take note that my port number for bootstrap-server is
> 9092. I made an error when i type the command
> /usr/bin/kafka-console-consumer --bootstrap-server
> EXTERNAL_IP:9092
>
> regards,
> Kim Tee
>
> On Mon, Apr 17, 2017 at 10:49 AM, Lim Kim Tee

> <stealth...@gmail.com <mailto:stealth...@gmail.com>>

> <http://kafka.producer.SyncProducer.li>ftedTree1$1(SyncProducer.scala:80)

> at
> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)
> at kafka.producer.SyncProducer.se

> <http://kafka.producer.SyncProducer.se>nd(SyncProducer.scala:124)

> <http://kafka.producer.SyncProducer.li>ftedTree1$1(SyncProducer.scala:80)

> at
> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)
> at kafka.producer.SyncProducer.se

> <http://kafka.producer.SyncProducer.se>nd(SyncProducer.scala:124)

> 192.168.4.156:2181 <http://192.168.4.156:2181>

> --describe --group console-consumer-69259
>
> this is the output
> Note: This will only show information about consumers
> that use ZooKeeper (not those using the Java consumer API).
>
>
> TOPIC PARTITION CURRENT-OFFSET
> LOG-END-OFFSET LAG CONSUMER-ID
>
>
> It seem like there is no output.
> I cant find the log for kafka stream.
>
> Please advise if I am doing correctly. Sorry for the
> lengthy post. Thanks.
>
> regards,
> Kim
>
>
> On Thu, Apr 13, 2017 at 11:44 PM, Matthias J. Sax

> <matt...@confluent.io <mailto:matt...@confluent.io>>

> wrote:
>
> Thanks for the update.
>
> It's hard to tell...
>
> - can you please try out bin/kafka-console-consumer
> script to see if
> you are able to connect to the brokers in the first
> place
> - do you see anything suspicious in the Kafka
> Streams logs?
> - please also check, if the Streams app shows app
> as a consumer group
> - you can use bin/kafka-consumer-groups script
> (Streams uses it's

> application ID as group.id <http://group.id> internally)

>
> Hope that this reveals what the problem might be.
>
> -Matthias
>
> On 4/12/17 7:11 PM, Lim Kim Tee wrote:
> > Hi Matthias,
> >
> > Sorry for the late reply as I just returned from an holiday.
> > I have deployed CP 3.2 on a single ubuntu 16.04 setup. So all the
> > packages from CP 3.2 are all in a single Ubuntu 16.04 since I wanted to
> > test out how Kafka works.
> >
> > My setup consists of the following:
> > 1, 1 x CP 3.2 in Ubuntu Server 16.04
> > 2. 1 x Windows 10 Eclipse Mars Java 1.8 to test out the Kafka Java API,
> > I have also tried using Ubuntu Desktop 16.04 Eclipse to see is it an OS
> > issue. So far it is the same.
> >
> > Please advise. Thanks.
> >
> > Regards,
> > Kim Tee
> >
> > On Sat, Apr 8, 2017 at 1:01 AM, Matthias J. Sax <matt...@confluent.io <mailto:matt...@confluent.io>

> > <mailto:matt...@confluent.io

> <mailto:matt...@confluent.io>>> wrote:
> >
> > Do you deploy CP3.2 locally for each setup? Or do you have one CP
> > deployment and just run Kafka Streams app in different environments?
> >
> >
> > -Matthias
> >
> > On 4/5/17 11:24 PM, Lim Kim Tee wrote:
> > > Hi I just want to clarify that the examples I have re-run is using
> > > Eclipse in Ubuntu 16.04 desktop.
> > > Following the link I provided from earlier, I have re-run in Eclipse on
> > > Windows.
> > >
> > > I have tried to run the examples in Ubuntu server 16.04, the OS i have
> > > deployed Confluent platform. It works.
> > > Just somehow I am unable to run the example using both of my WIndows and
> > > Ubuntu Desktop.
> > >
> > > I would like to ask if there is other thing I might have missed out? Thanks.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "Confluent Platform" group.
> > > To unsubscribe from this group and stop receiving emails from it, send

> > > an email to confluent-platf...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>
> >
> <mailto:confluent-platform%2Bunsu...@googlegroups.com
> <mailto:confluent-platform%252Buns...@googlegroups.com>>
> > >
> <mailto:confluent-platf...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>
> >
> <mailto:confluent-platform%2Bunsu...@googlegroups.com
> <mailto:confluent-platform%252Buns...@googlegroups.com>>>.

> > > To post to this group, send email to confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>
> > <mailto:confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>>
> > > <mailto:confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>
> > <mailto:confluent...@googlegroups.com

> <mailto:confluent...@googlegroups.com>>>.

> > confluent-platf...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>
> >
> <mailto:confluent-platform%2Bunsu...@googlegroups.com
> <mailto:confluent-platform%252Buns...@googlegroups.com>>.

> > To post to this group, send email to

> > confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>
> > <mailto:confluent...@googlegroups.com

> <mailto:confluent...@googlegroups.com>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io
> <https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io>
> >
> <https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io
> <https://groups.google.com/d/msgid/confluent-platform/eb8aa9ba-faa9-2e01-6be6-d53b83ed9df4%40confluent.io>>.
> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
> > <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>.
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Confluent Platform" group.
> > To unsubscribe from this group and stop receiving emails from it, send

> <mailto:confluent...@googlegroups.com>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com
> <https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com>
> >

> <https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/confluent-platform/CACN1X6LQemG%3DHmtvqMOV%3D8EGyWL7nw77cU0gpsCFc-mcFdxQvA%40mail.gmail.com?utm_medium=email&utm_source=footer>>.

> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed
> to a topic in the Google Groups "Confluent Platform"
> group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe
> <https://groups.google.com/d/topic/confluent-platform/DxKkKn2cQBE/unsubscribe>.
> To unsubscribe from this group and all its topics,
> send an email to

> confluent-platf...@googlegroups.com
> <mailto:confluent-platform%2Bunsu...@googlegroups.com>.

> To post to this group, send email to

> confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/confluent-platform/2f5ef71d-df58-e5a3-aaa8-e5d68a4c0f7d%40confluent.io
> <https://groups.google.com/d/msgid/confluent-platform/2f5ef71d-df58-e5a3-aaa8-e5d68a4c0f7d%40confluent.io>.

> For more options, visit
> https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/confluent-platform/CACN1X6Lpn1Ta%2B6Dv7VmPoWf_wb%2BK_O5Wp4qsRDhYO2gnUc1r8w%40mail.gmail.com
> <https://groups.google.com/d/msgid/confluent-platform/CACN1X6Lpn1Ta%2B6Dv7VmPoWf_wb%2BK_O5Wp4qsRDhYO2gnUc1r8w%40mail.gmail.com?utm_medium=email&utm_source=footer>.