Unable to load data to indexService

[tao...@yahoo-inc.com]

Hi, Nishant and Fangjin:

Open another topic to go on to fix the issue I met.

Here is the configuration about overlord:

druid.host=<hostname>

druid.port=8087

#druid.service=overlord

druid.service=druid/v1/indexer

druid.zk.service.host=localhost

druid.db.connector.connectURI=jdbc:mysql://localhost:3306/druid

druid.db.connector.user=druid

druid.db.connector.password=diurd

druid.selectors.indexing.serviceName=druid:v1:indexer

druid.indexer.queue.startDelay=PT0M

druid.indexer.runner.javaOpts="-server -Xmx1g -Xms1g -XX:NewSize=256m -XX:MaxNewSize=256m -XX:MaxDirectMemorySize=536870912  -XX:+PrintGCDetails -XX:+PrintGCTimeStamps"

druid.indexer.runner.startPort=8088

druid.indexer.fork.property.druid.computation.buffer.size=268435456

#druid.indexer.storage.type=db

druid.db.connector.useValidationQuery=true

When load data, some exceptions log from "console.html" are described as blow:

2014-04-10 04:27:47,171 INFO [main] io.druid.server.initialization.PropertiesModule - Loading properties from runtime.properties
2014-04-10 04:27:47,249 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.0.1.Final
2014-04-10 04:27:47,933 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[], localRepository='/home/taoluo/.m2/repository', remoteRepositories=[http://repo1.maven.org/maven2/, https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local]}]
2014-04-10 04:27:49,365 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class com.metamx.emitter.core.LoggingEmitterConfig] from props[druid.emitter.logging.] as [LoggingEmitterConfig{loggerClass='com.metamx.emitter.core.LoggingEmitter', logLevel='info'}]
2014-04-10 04:27:49,485 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@31174ed3]
2014-04-10 04:27:49,510 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[]}]
2014-04-10 04:27:49,562 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.DruidNode] from props[druid.] as [DruidNode{serviceName='druid/v1/indexer', host='hostname:8088', port=8088}]
2014-04-10 04:27:49,679 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.TaskConfig] from props[druid.indexer.task.] as [io.druid.indexing.common.config.TaskConfig@60bb132e]
2014-04-10 04:27:49,695 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.HttpClientModule$DruidHttpClientConfig] from props[druid.global.http.] as [io.druid.guice.HttpClientModule$DruidHttpClientConfig@724d05fa]
2014-04-10 04:27:49,789 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.indexing.IndexingServiceSelectorConfig] from props[druid.selectors.indexing.] as [io.druid.client.indexing.IndexingServiceSelectorConfig@5ad7eb94]
2014-04-10 04:27:49,792 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [localhost] for [druid.zk.service.host] on [io.druid.curator.CuratorConfig#getZkHosts()]
2014-04-10 04:27:49,795 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning default value [30000] for [druid.zk.service.sessionTimeoutMs] on [io.druid.curator.CuratorConfig#getZkSessionTimeoutMs()]
2014-04-10 04:27:49,795 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning default value [false] for [druid.curator.compress] on [io.druid.curator.CuratorConfig#enableCompression()]
2014-04-10 04:27:49,905 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29
2014-04-10 04:27:50,024 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.CuratorDiscoveryConfig] from props[druid.discovery.curator.] as [io.druid.server.initialization.CuratorDiscoveryConfig@2ca258c2]
2014-04-10 04:27:50,253 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.RetryPolicyConfig] from props[druid.peon.taskActionClient.retry.] as [io.druid.indexing.common.RetryPolicyConfig@1c2859eb]
2014-04-10 04:27:50,259 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.segment.loading.LocalDataSegmentPusherConfig] from props[druid.storage.] as [io.druid.segment.loading.LocalDataSegmentPusherConfig@1316f155]
2014-04-10 04:27:50,282 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.DruidServerConfig] from props[druid.server.] as [io.druid.client.DruidServerConfig@544e9892]
2014-04-10 04:27:50,282 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.liveSegmentsPath] on [io.druid.server.initialization.ZkPathsConfig#getLiveSegmentsPath()]
2014-04-10 04:27:50,283 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.base] on [io.druid.server.initialization.ZkPathsConfig#getZkBasePath()]
2014-04-10 04:27:50,284 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.announcementsPath] on [io.druid.server.initialization.ZkPathsConfig#getAnnouncementsPath()]
2014-04-10 04:27:50,284 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.servedSegmentsPath] on [io.druid.server.initialization.ZkPathsConfig#getServedSegmentsPath()]
2014-04-10 04:27:50,284 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.coordinatorPath] on [io.druid.server.initialization.ZkPathsConfig#getCoordinatorPath()]
2014-04-10 04:27:50,284 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.propertiesPath] on [io.druid.server.initialization.ZkPathsConfig#getPropertiesPath()]
2014-04-10 04:27:50,284 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.indexer.announcementsPath] on [io.druid.server.initialization.ZkPathsConfig#getIndexerAnnouncementPath()]
2014-04-10 04:27:50,285 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.indexer.tasksPath] on [io.druid.server.initialization.ZkPathsConfig#getIndexerTaskPath()]
2014-04-10 04:27:50,285 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.loadQueuePath] on [io.druid.server.initialization.ZkPathsConfig#getLoadQueuePath()]
2014-04-10 04:27:50,285 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.indexer.statusPath] on [io.druid.server.initialization.ZkPathsConfig#getIndexerStatusPath()]
2014-04-10 04:27:50,286 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.indexer.leaderLatchPath] on [io.druid.server.initialization.ZkPathsConfig#getIndexerLeaderLatchPath()]
2014-04-10 04:27:50,358 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.BatchDataSegmentAnnouncerConfig] from props[druid.announcer.] as [io.druid.server.initialization.BatchDataSegmentAnnouncerConfig@44c6f613]
2014-04-10 04:27:50,366 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.coordination.DataSegmentAnnouncerProvider] from props[druid.announcer.] as [io.druid.server.coordination.LegacyDataSegmentAnnouncerProvider@1393abe4]
2014-04-10 04:27:50,374 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.client.ServerInventoryViewProvider] from props[druid.announcer.] as [io.druid.client.SingleServerInventoryProvider@7e768122]
2014-04-10 04:27:50,384 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.QueryConfig] from props[druid.query.] as [io.druid.query.QueryConfig@3bd24471]
2014-04-10 04:27:50,396 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.search.search.SearchQueryConfig] from props[druid.query.search.] as [io.druid.query.search.search.SearchQueryConfig@7f85a53b]
2014-04-10 04:27:50,412 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.groupby.GroupByQueryConfig] from props[druid.query.groupBy.] as [io.druid.query.groupby.GroupByQueryConfig@1334918e]
2014-04-10 04:27:50,412 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [268435456] for [druid.computation.buffer.size] on [io.druid.server.DruidProcessingConfig#intermediateComputeSizeBytes()]
2014-04-10 04:27:50,413 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning default value [1] for [${base_path}.numThreads] on [com.metamx.common.concurrent.ExecutorServiceConfig#getNumThreads()]
2014-04-10 04:27:50,414 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning default value [processing-%s] for [${base_path}.formatString] on [com.metamx.common.concurrent.ExecutorServiceConfig#getFormatString()]
2014-04-10 04:27:50,428 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.log.RequestLoggerProvider] from props[druid.request.logging.] as [io.druid.server.log.NoopRequestLoggerProvider@368a8022]
2014-04-10 04:27:50,432 ERROR [main] io.druid.cli.CliPeon - Error when starting up.  Failing.
com.google.inject.ProvisionException: Guice provision errors:

1) Not enough direct memory.  Please adjust -XX:MaxDirectMemorySize or druid.computation.buffer.size: maxDirectMemory[508,887,040], memoryNeeded[536,870,912], druid.computation.buffer.size[268,435,456], numThreads[1]
  at io.druid.guice.DruidProcessingModule.getIntermediateResultsPool(DruidProcessingModule.java:76)
  at io.druid.guice.DruidProcessingModule.getIntermediateResultsPool(DruidProcessingModule.java:76)
  while locating io.druid.collections.StupidPool<java.nio.ByteBuffer> annotated with @io.druid.guice.annotations.Global()
    for parameter 1 at io.druid.query.groupby.GroupByQueryEngine.<init>(GroupByQueryEngine.java:76)
  at io.druid.guice.QueryRunnerFactoryModule.configure(QueryRunnerFactoryModule.java:71)
  while locating io.druid.query.groupby.GroupByQueryEngine
    for parameter 0 at io.druid.query.groupby.GroupByQueryRunnerFactory.<init>(GroupByQueryRunnerFactory.java:61)
  at io.druid.guice.QueryRunnerFactoryModule.configure(QueryRunnerFactoryModule.java:68)
  while locating io.druid.query.groupby.GroupByQueryRunnerFactory
  while locating io.druid.query.QueryRunnerFactory annotated with @com.google.inject.multibindings.Element(setName=,uniqueId=22, type=MAPBINDER)
  at io.druid.guice.DruidBinders.queryRunnerFactoryBinder(DruidBinders.java:38)
  while locating java.util.Map<java.lang.Class<? extends io.druid.query.Query>, io.druid.query.QueryRunnerFactory>
    for parameter 0 at io.druid.query.DefaultQueryRunnerFactoryConglomerate.<init>(DefaultQueryRunnerFactoryConglomerate.java:36)
  while locating io.druid.query.DefaultQueryRunnerFactoryConglomerate
  at io.druid.guice.StorageNodeModule.configure(StorageNodeModule.java:52)
  while locating io.druid.query.QueryRunnerFactoryConglomerate
    for parameter 9 at io.druid.indexing.common.TaskToolboxFactory.<init>(TaskToolboxFactory.java:78)
  at io.druid.cli.CliPeon$1.configure(CliPeon.java:124)
  while locating io.druid.indexing.common.TaskToolboxFactory
    for parameter 0 at io.druid.indexing.overlord.ThreadPoolTaskRunner.<init>(ThreadPoolTaskRunner.java:68)
  at io.druid.cli.CliPeon$1.configure(CliPeon.java:150)
  while locating io.druid.indexing.overlord.ThreadPoolTaskRunner
  while locating io.druid.query.QuerySegmentWalker
    for parameter 2 at io.druid.server.QueryResource.<init>(QueryResource.java:75)
  while locating io.druid.server.QueryResource

1 error
	at com.google.inject.internal.InjectorImpl$3.get(InjectorImpl.java:1014)
	at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1036)
	at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:134)
	at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:72)
	at io.druid.cli.CliPeon.run(CliPeon.java:202)
	at io.druid.cli.Main.main(Main.java:91)

And the overlord running log is attached.

I haven't found some keywords about "memory" in running log.

Could you help me to find the reason? 

Thank you very much.

[Nishant Bangarwa]

Hi Tao, 

Looks like the JsonConfigProvider is parsing the javaOpts as empty String when you add quotes around the property in runtime.properties 

Can you try running by removing quotes from your runtime.props as a workaround ?

e.g. 

druid.indexer.runner.javaOpts=-server -Xmx1g -Xms1g -XX:NewSize=256m -XX:MaxNewSize=256m -XX:MaxDirectMemorySize=536870912  -XX:+PrintGCDetails -XX:+PrintGCTimeStamps

--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/de4fd20a-2cf1-4284-b45b-c5cf59ad3fbb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Nishant Software Engineer | METAMARKETS m  +91-9729200044 nishant....@metamarkets.com

[tao...@yahoo-inc.com]

Oh, yes!!! succeed! Thank you very much! ^_^