Kafka Connect throws exception with kafka+ssl enabled.

164 views
Skip to first unread message

Ashish Sachdeva

unread,
Sep 11, 2017, 8:23:40 AM9/11/17
to Confluent Platform
I enabled SSL on my kafka broker as explained here:
http://docs.confluent.io/current/kafka/ssl.html

Added the following fields in server.properties in kafka broker:
listeners=PLAINTEXT://localhost:9092,SSL://localhost:9093
advertised.listeners=PLAINTEXT://localhost:9092,SSL://localhost:9093

ssl.keystore.location=<path_to_kafka.server.keystore.jks>
ssl.keystore.password=<pwd>
ssl.key.password=<pwd>
ssl.truststore.location=<path_to_kafka.server.truststore.jks>
ssl.truststore.password=<pwd>
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1

and following in connect-distributed.properties in kafka connect:
bootstrap.servers=<kafka broker>:9093

security.protocol=SSL
ssl.truststore.location=<path_to_kafka.client.truststore.jks>
ssl.truststore.password=<pwd>

after this when I start kafka connect process, I am getting the following exception:
[2017-09-11 11:49:12,198] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:56)
[2017-09-11 11:54:15,973] ERROR Uncaught exception in herder work thread, exiting:  (org.apache.kafka.connect.runtime.distributed.DistributedHerder:207)
org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
[2017-09-11 11:54:15,975] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:66)
[2017-09-11 11:54:15,975] INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer:154)


I have given permission to access ports 9092 & 9093 on kafka broker from kafka connect node. 
If I remove all the ssl parameters from broker as well as connect side, everything seems to be working fine.

Really appreciate any pointers on what I am doing wrong or if I am missing something?

- Ashish

Randall Hauch

unread,
Sep 11, 2017, 1:41:48 PM9/11/17
to confluent...@googlegroups.com
The TLS/SSL properties must be set on the worker's producer and consumer. http://docs.confluent.io/current/connect/security.html

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/db0462e1-b108-448d-a91b-d6c6a71420c3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ashish Sachdeva

unread,
Sep 18, 2017, 10:53:00 AM9/18/17
to Confluent Platform
Actual issue turns out to be how listeners/advertised listeners are specified in the server.properties file:
instead of localhost it has to be public dns of broker. 

@Randall your url actually helped in solving another ssl exception I was getting after replacing localhost above. Thanks.!
Reply all
Reply to author
Forward
0 new messages