Druid Elasticsearch Extension

[11512...@qq.com]

hi, all:

I have a question when I want to finish a elasticsearch extension as as druid-hdfs-storage extension  for druid v0.9.0.

I modify the _common/common.runtime.properties file,include the following config:

#druid.extensions.loadList=["druid-hdfs-storage"]

druid.extensions.loadList=["druid-es-storage"]

# For HDFS:

#druid.storage.type=hdfs

#druid.storage.storageDirectory=hdfs://10.0.3.49:9000/druid

#For ES:

druid.storage.type=es

druid.storage.storageDirectory=/druid/segments

But, when I start the historcial node, generate a exception, If I want to add a extesion, where I want to config???

My telephone is 18514477499, Who can give me some help??? Thanke you!

2016-06-10T07:27:43,390 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.metadata.SegmentMetadataQueryConfig] from props[druid.query.segmentMetadata.] as [io.druid.query.metadata.SegmentMetadataQueryConfig@6579b984]

2016-06-10T07:27:43,398 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.groupby.GroupByQueryConfig] from props[druid.query.groupBy.] as [io.druid.query.groupby.GroupByQueryConfig@6dad0af1]

2016-06-10T07:27:43,432 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.topn.TopNQueryConfig] from props[druid.query.topN.] as [io.druid.query.topn.TopNQueryConfig@3e767b19]

2016-06-10T07:27:43,446 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.cache.CacheConfig] from props[druid.historical.cache.] as [io.druid.client.cache.CacheConfig@544d0be5]

2016-06-10T07:27:43,454 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.log.RequestLoggerProvider] from props[druid.request.logging.] as [io.druid.server.log.NoopRequestLoggerProvider@39683cf4]

2016-06-10T07:27:43,455 ERROR [main] io.druid.cli.CliHistorical - Error when starting up.  Failing.

com.google.inject.ProvisionException: Guice provision errors:

1) druid.segmentCache.locations - may not be empty

  at io.druid.guice.JsonConfigProvider.bind(JsonConfigProvider.java:131)

  at io.druid.guice.JsonConfigProvider.bind(JsonConfigProvider.java:131)

  while locating com.google.common.base.Supplier<io.druid.segment.loading.SegmentLoaderConfig>

  at io.druid.guice.JsonConfigProvider.bind(JsonConfigProvider.java:132)

  while locating io.druid.segment.loading.SegmentLoaderConfig

    for parameter 1 at io.druid.segment.loading.SegmentLoaderLocalCacheManager.<init>(SegmentLoaderLocalCacheManager.java:59)

  while locating io.druid.segment.loading.SegmentLoaderLocalCacheManager

  at io.druid.guice.LocalDataStorageDruidModule.configure(LocalDataStorageDruidModule.java:53)

  while locating io.druid.segment.loading.SegmentLoader

    for parameter 0 at io.druid.server.coordination.ServerManager.<init>(ServerManager.java:106)

  at io.druid.cli.CliHistorical$1.configure(CliHistorical.java:81)

  while locating io.druid.server.coordination.ServerManager

  at io.druid.cli.CliHistorical$1.configure(CliHistorical.java:85)

  while locating io.druid.query.QuerySegmentWalker

    for parameter 3 at io.druid.server.QueryResource.<init>(QueryResource.java:95)

  while locating io.druid.server.QueryResource

1 error

        at com.google.inject.internal.InjectorImpl$3.get(InjectorImpl.java:1014) ~[guice-4.0-beta.jar:?]

        at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1036) ~[guice-4.0-beta.jar:?]

        at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:153) ~[druid-api-0.3.16.jar:0.9.0]

        at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:91) [druid-services-0.9.0.jar:0.9.0]

        at io.druid.cli.ServerRunnable.run(ServerRunnable.java:40) [druid-services-0.9.0.jar:0.9.0]

        at io.druid.cli.Main.main(Main.

[Gian Merlino]

This error "druid.segmentCache.locations - may not be empty" seems unrelated to your extension. Probably the conf/historical directory is missing from the classpath, so CliHistorical can't read the runtime.properties in there. Those runtime.properties should have something set for druid.segmentCache.locations.

Gian

--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/0e507899-7d0e-47d0-9de0-203f5ff122ff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[11512...@qq.com]

Thank you very much，I have resolveed it，the reason is that the Properties.class is override by me， Now I have a new question, When I user hdfs, after the segemtn is created, it will be pushed to hdfs-storage, in the hdfs-storage push method, you can get a directory consisit of druid data file,version file..., Can I get my real data from the file. How can I parse the file.

My purpose is get  the read data ,then save to elasticsearch........

在 2016年6月10日星期五 UTC+8下午9:43:15，Gian Merlino写道：

[Fangjin Yang]

In addition to the discussion here, I also want to add that there are folks currently working in supporting Lucene as part of Druid:

https://github.com/druid-io/druid/issues/2965