Docker image for druid kafka indexing service

[Ivan Nikitsenka]

Hello

I'm working on dockerfile for building image with ready to use druid kafka indexing service. I'm using https://github.com/druid-io/docker-druid as basis.

Here is my current changes https://github.com/nikitsenka/docker-druid-kafka/pull/1/files

Image build works successfully, but having an ""Could not resolve type id 'kafka' into a subtype" error when running it in docker

 curl -X POST -H 'Content-Type: application/json' -d @supervisor-spec.json http://172.17.0.3:8081/druid/indexer/v1/supervisor

{"error":"Could not resolve type id 'kafka' into a subtype of [simple type, class io.druid.indexing.overlord.supervisor.SupervisorSpec]\n at [Source: HttpInputOverHTTP@274de357[c=1175,q=1,[0]=EOF,s=STREAM]; line: 1, column: 4]"}

The error log

druid_1          | 2018-05-03T12:20:56,783 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/local/druid/extensions/druid-kafka-indexing-service/lz4-1.3.0.jar]

druid_1          | 2018-05-03T12:20:56,784 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/local/druid/extensions/druid-kafka-indexing-service/slf4j-api-1.6.4.jar]

druid_1          | 2018-05-03T12:20:56,784 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/local/druid/extensions/druid-kafka-indexing-service/snappy-java-1.1.2.6.jar]

druid_1          | 2018-05-03T12:20:57,092 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-kafka-indexing-service] for class [interface io.druid.initialization.DruidModule]

druid_1          | Error: Could not find or load main class "druid-kafka-indexing-service"]

druid_1          | Error: Could not find or load main class "druid-kafka-indexing-service"]

ENV DRUID_VERSION 0.12.0

ENV ZOOKEEPER_VERSION 3.4.12

ENV KAFKA_VERSION 1.1.0

full changes can be found here https://github.com/nikitsenka/docker-druid-kafka/pull/1/files#diff-3254677a7917c6c01f55212f86c57fbfR5

Any advice or ideas how to fix will be appreciated.

If anyone has ready to use kafka-indexting-service configuration supervisord.conf  or docker image please share.

Thank you

Ivan

[Gian Merlino]

Hi Ivan,

That probably means the druid-kafka-indexing-service extension isn't being loaded properly. The "Error: Could not find or load main class druid-kafka-indexing-service" bit looks like there is some problem generating the command line -- maybe something off by one arg. Try double checking all the whitespace, maybe remove whitespace in druid.extensions.loadList.

Gian

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+unsubscribe@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/24fcabd4-0e91-4a7e-b424-c0a8104a879f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ivan Nikitsenka]

Gian,

Yes, your truth

The problem was in whitespace -Ddruid.extensions.loadList=[\"mysql-metadata-storage\", \"druid-kafka-indexing-service\"]

Thank you

Gian

To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.