Fail to finish Hadoop-based Batch Ingestion

535 views
Skip to first unread message

Yi

unread,
Aug 7, 2016, 9:50:30 AM8/7/16
to Druid User
Hello,

I try to setup a cluster with CDH 5.6,  (rebulid Druid with hadoop2.6.0  and config  "mapreduce.job.user.classpath.first" : "true"  


but the mapreduce job seems never stop.

config for common.runtime.properties:

druid.extensions.loadList=["druid-histogram",  "mysql-metadata-storage", "druid-hdfs-storage"]
druid.zk.service.host=hadoop02
druid.metadata.storage.type=mysql
druid.metadata.storage.connector.connectURI=jdbc:mysql://xxx:3306/druid
druid.metadata.storage.connector.user=xxx
druid.metadata.storage.connector.password=xxxx
druid.storage.type=hdfs
druid.storage.storageDirectory=hdfs://hadoop02/tmp/druid/localStorage
druid.cache.type=local
druid.cache.sizeInBytes=10000000
druid.selectors.indexing.serviceName=overlord
druid.selectors.coordinator.serviceName=coordinator
druid.emitter=logging
druid.indexer.logs.type=hdfs
druid.indexer.logs.directory=/tmp/indexing-logs


below is log :


2016-08-07T13:09:26,047 INFO [main] io.druid.guice.PropertiesModule - Loading properties from runtime.properties
Aug 07, 2016 1:09:26 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 5.1.3.Final
2016-08-07T13:09:26,539 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, directory='extensions', hadoopDependenciesDir='hadoop-dependencies', hadoopContainerDruidClasspath='null', loadList=[druid-histogram, mysql-metadata-storage, druid-hdfs-storage]}]
2016-08-07T13:09:26,544 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-histogram] for class [io.druid.cli.CliCommandCreator]
2016-08-07T13:09:26,553 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-histogram/druid-histogram-0.9.1.1.jar]
2016-08-07T13:09:26,554 INFO [main] io.druid.initialization.Initialization - Loading extension [mysql-metadata-storage] for class [io.druid.cli.CliCommandCreator]
2016-08-07T13:09:26,554 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/mysql-metadata-storage/mysql-connector-java-5.1.38.jar]
2016-08-07T13:09:26,554 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/mysql-metadata-storage/mysql-metadata-storage-0.9.1.1.jar]
2016-08-07T13:09:26,555 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-hdfs-storage] for class [io.druid.cli.CliCommandCreator]
2016-08-07T13:09:26,556 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xercesImpl-2.9.1.jar]
2016-08-07T13:09:26,556 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/api-asn1-api-1.0.0-M20.jar]
2016-08-07T13:09:26,556 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/stax-api-1.0-2.jar]
2016-08-07T13:09:26,557 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-common-2.6.0.jar]
2016-08-07T13:09:26,557 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-compress-1.4.1.jar]
2016-08-07T13:09:26,557 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-recipes-2.10.0.jar]
2016-08-07T13:09:26,557 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-hdfs-2.6.0.jar]
2016-08-07T13:09:26,557 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xz-1.0.jar]
2016-08-07T13:09:26,557 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/druid-hdfs-storage-0.9.1.1.jar]
2016-08-07T13:09:26,557 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-beanutils-1.7.0.jar]
2016-08-07T13:09:26,558 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/servlet-api-2.5.jar]
2016-08-07T13:09:26,558 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/apacheds-kerberos-codec-2.0.0-M15.jar]
2016-08-07T13:09:26,558 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-app-2.6.0.jar]
2016-08-07T13:09:26,558 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-net-3.1.jar]
2016-08-07T13:09:26,558 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jersey-client-1.9.jar]
2016-08-07T13:09:26,558 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-configuration-1.6.jar]
2016-08-07T13:09:26,559 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/htrace-core-3.0.4.jar]
2016-08-07T13:09:26,559 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-digester-1.8.jar]
2016-08-07T13:09:26,559 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-client-2.6.0.jar]
2016-08-07T13:09:26,559 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jaxb-api-2.2.2.jar]
2016-08-07T13:09:26,559 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-framework-2.10.0.jar]
2016-08-07T13:09:26,559 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-common-2.6.0.jar]
2016-08-07T13:09:26,559 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-shuffle-2.6.0.jar]
2016-08-07T13:09:26,560 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/guava-16.0.1.jar]
2016-08-07T13:09:26,560 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-core-2.6.0.jar]
2016-08-07T13:09:26,560 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xmlenc-0.52.jar]
2016-08-07T13:09:26,560 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xml-apis-1.3.04.jar]
2016-08-07T13:09:26,560 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-api-2.6.0.jar]
2016-08-07T13:09:26,560 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/netty-3.6.2.Final.jar]
2016-08-07T13:09:26,560 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-common-2.6.0.jar]
2016-08-07T13:09:26,560 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/leveldbjni-all-1.8.jar]
2016-08-07T13:09:26,561 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/avro-1.7.4.jar]
2016-08-07T13:09:26,561 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-client-2.10.0.jar]
2016-08-07T13:09:26,561 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/gson-2.2.4.jar]
2016-08-07T13:09:26,561 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-server-common-2.6.0.jar]
2016-08-07T13:09:26,561 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-client-2.6.0.jar]
2016-08-07T13:09:26,561 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/api-util-1.0.0-M20.jar]
2016-08-07T13:09:26,561 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-math3-3.6.1.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-auth-2.6.0.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-jobclient-2.6.0.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/apacheds-i18n-2.0.0-M15.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/paranamer-2.3.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-beanutils-core-1.8.0.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jackson-jaxrs-1.9.13.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-collections-3.2.1.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/snappy-java-1.0.4.1.jar]
2016-08-07T13:09:26,562 INFO [main] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jackson-xc-1.9.13.jar]
2016-08-07T13:09:26,717 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-histogram] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:26,719 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.query.aggregation.histogram.ApproximateHistogramDruidModule] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:26,719 INFO [main] io.druid.initialization.Initialization - Loading extension [mysql-metadata-storage] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:26,720 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.metadata.storage.mysql.MySQLMetadataStorageModule] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:26,720 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-hdfs-storage] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:26,722 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.storage.hdfs.HdfsStorageDruidModule] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:27,931 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-08-07T13:09:28,241 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class com.metamx.emitter.core.LoggingEmitterConfig] from props[druid.emitter.logging.] as [LoggingEmitterConfig{loggerClass='com.metamx.emitter.core.LoggingEmitter', logLevel='info'}]
2016-08-07T13:09:28,301 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@6e0c6a7a]
2016-08-07T13:09:28,309 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[]}]
2016-08-07T13:09:28,334 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.DruidNode] from props[druid.] as [DruidNode{serviceName='overlord', host='hadoop02', port=8100}]
2016-08-07T13:09:28,336 INFO [main] io.druid.server.metrics.MetricsModule - Adding monitor[io.druid.query.ExecutorServiceMonitor@317a118b]
2016-08-07T13:09:28,513 INFO [main] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties
2016-08-07T13:09:28,514 INFO [main] io.druid.guice.PropertiesModule - Loading properties from runtime.properties
2016-08-07T13:09:28,525 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, directory='extensions', hadoopDependenciesDir='hadoop-dependencies', hadoopContainerDruidClasspath='null', loadList=[druid-histogram, mysql-metadata-storage, druid-hdfs-storage]}]
2016-08-07T13:09:28,527 INFO [main] io.druid.server.metrics.MetricsModule - Adding monitor[io.druid.server.initialization.jetty.JettyServerModule$JettyMonitor@2687725a]
2016-08-07T13:09:28,535 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.log.StartupLoggingConfig] from props[druid.startup.logging.] as [io.druid.server.log.StartupLoggingConfig@5434e40c]
2016-08-07T13:09:28,536 INFO [main] io.druid.cli.CliPeon - Starting up with processors[8], memory[444,071,936].
2016-08-07T13:09:28,549 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.TaskConfig] from props[druid.indexer.task.] as [io.druid.indexing.common.config.TaskConfig@47d023b7]
2016-08-07T13:09:28,553 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.http.DruidHttpClientConfig] from props[druid.global.http.] as [io.druid.guice.http.DruidHttpClientConfig@26fb4d06]
2016-08-07T13:09:28,607 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.indexing.IndexingServiceSelectorConfig] from props[druid.selectors.indexing.] as [io.druid.client.indexing.IndexingServiceSelectorConfig@1921994e]
2016-08-07T13:09:28,610 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.curator.CuratorConfig] from props[druid.zk.service.] as [io.druid.curator.CuratorConfig@3a2d3909]
2016-08-07T13:09:28,615 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29
2016-08-07T13:09:28,639 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.CuratorDiscoveryConfig] from props[druid.discovery.curator.] as [io.druid.server.initialization.CuratorDiscoveryConfig@4ee25d80]
2016-08-07T13:09:28,781 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.RetryPolicyConfig] from props[druid.peon.taskActionClient.retry.] as [io.druid.indexing.common.RetryPolicyConfig@36c2d629]
2016-08-07T13:09:28,783 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.storage.hdfs.HdfsDataSegmentPusherConfig] from props[druid.storage.] as [io.druid.storage.hdfs.HdfsDataSegmentPusherConfig@3b95a6db]
2016-08-07T13:09:28,784 INFO [main] io.druid.storage.hdfs.HdfsDataSegmentPusher - Configured HDFS as deep storage
2016-08-07T13:09:28,789 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.DruidServerConfig] from props[druid.server.] as [io.druid.client.DruidServerConfig@2c7db926]
2016-08-07T13:09:28,792 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.BatchDataSegmentAnnouncerConfig] from props[druid.announcer.] as [io.druid.server.initialization.BatchDataSegmentAnnouncerConfig@7209ffb5]
2016-08-07T13:09:28,795 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ZkPathsConfig] from props[druid.zk.paths.] as [io.druid.server.initialization.ZkPathsConfig@58d3f4be]
2016-08-07T13:09:28,801 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.coordination.DataSegmentAnnouncerProvider] from props[druid.announcer.] as [io.druid.server.coordination.BatchDataSegmentAnnouncerProvider@4cb702ce]
2016-08-07T13:09:28,803 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.coordinator.CoordinatorSelectorConfig] from props[druid.selectors.coordinator.] as [io.druid.client.coordinator.CoordinatorSelectorConfig@430b2699]
2016-08-07T13:09:28,805 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.segment.realtime.plumber.CoordinatorBasedSegmentHandoffNotifierConfig] from props[druid.segment.handoff.] as [io.druid.segment.realtime.plumber.CoordinatorBasedSegmentHandoffNotifierConfig@3b55dd15]
2016-08-07T13:09:28,808 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [53687] for [druid.processing.buffer.sizeBytes] on [io.druid.query.DruidProcessingConfig#intermediateComputeSizeBytes()]
2016-08-07T13:09:28,809 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [1] for [druid.processing.numThreads] on [io.druid.query.DruidProcessingConfig#getNumThreads()]
2016-08-07T13:09:28,810 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.computation.buffer.poolCacheMaxCount, ${base_path}.buffer.poolCacheMaxCount] on [io.druid.query.DruidProcessingConfig#poolCacheMaxCount()]
2016-08-07T13:09:28,811 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.columnCache.sizeBytes] on [io.druid.query.DruidProcessingConfig#columnCacheSizeBytes()]
2016-08-07T13:09:28,811 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.fifo] on [io.druid.query.DruidProcessingConfig#isFifo()]
2016-08-07T13:09:28,811 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning default value [processing-%s] for [${base_path}.formatString] on [com.metamx.common.concurrent.ExecutorServiceConfig#getFormatString()]
2016-08-07T13:09:28,885 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.search.search.SearchQueryConfig] from props[druid.query.search.] as [io.druid.query.search.search.SearchQueryConfig@46e3559f]
2016-08-07T13:09:28,887 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.metadata.SegmentMetadataQueryConfig] from props[druid.query.segmentMetadata.] as [io.druid.query.metadata.SegmentMetadataQueryConfig@2d0b05]
2016-08-07T13:09:28,890 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.groupby.GroupByQueryConfig] from props[druid.query.groupBy.] as [io.druid.query.groupby.GroupByQueryConfig@6bedd4e4]
2016-08-07T13:09:28,907 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.topn.TopNQueryConfig] from props[druid.query.topN.] as [io.druid.query.topn.TopNQueryConfig@6ccac6f4]
2016-08-07T13:09:28,914 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.client.cache.CacheProvider] from props[druid.cache.] as [io.druid.client.cache.LocalCacheProvider@66223d94]
2016-08-07T13:09:28,920 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.cache.CacheConfig] from props[druid.realtime.cache.] as [io.druid.client.cache.CacheConfig@61ae0d43]
2016-08-07T13:09:28,935 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ServerConfig] from props[druid.server.http.] as [ServerConfig{numThreads=40, maxIdleTime=PT5M}]
2016-08-07T13:09:28,936 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.log.RequestLoggerProvider] from props[druid.request.logging.] as [io.druid.server.log.NoopRequestLoggerProvider@e09f1b6]
2016-08-07T13:09:28,939 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.security.AuthConfig] from props[druid.auth.] as [AuthConfig{enabled=false}]
2016-08-07T13:09:28,947 INFO [main] org.eclipse.jetty.util.log - Logging initialized @3594ms
2016-08-07T13:09:29,029 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.lookup.LookupConfig] from props[druid.lookup.] as [LookupConfig{snapshotWorkingDir=''}]
2016-08-07T13:09:29,033 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.lookup.LookupListeningAnnouncerConfig] from props[druid.lookup.] as [ListeningAnnouncerConfig{listenersPath='/druid/listeners'}]
2016-08-07T13:09:29,034 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.emitter.core.LoggingEmitter.start()] on object[com.metamx.emitter.core.LoggingEmitter@9accff0].
2016-08-07T13:09:29,035 INFO [main] com.metamx.emitter.core.LoggingEmitter - Start: started [true]
2016-08-07T13:09:29,035 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.start()] on object[io.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner@1039bfc4].
2016-08-07T13:09:29,035 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.emitter.service.ServiceEmitter.start()] on object[com.metamx.emitter.service.ServiceEmitter@7f85217c].
2016-08-07T13:09:29,035 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.metrics.MonitorScheduler.start()] on object[com.metamx.metrics.MonitorScheduler@1fd7a37].
2016-08-07T13:09:29,039 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.http.client.NettyHttpClient.start()] on object[com.metamx.http.client.NettyHttpClient@73afe2b7].
2016-08-07T13:09:29,039 INFO [main] io.druid.curator.CuratorModule - Starting Curator
2016-08-07T13:09:29,039 INFO [main] org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:host.name=hadoop02
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_77
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-8-oracle/jre
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=conf/druid/_common:conf/druid/overlord:lib/druid-indexing-hadoop-0.9.1.1.jar:lib/mapdb-1.0.8.jar:lib/maven-model-3.1.1.jar:lib/tesla-aether-0.0.5.jar:lib/aws-java-sdk-codecommit-1.10.21.jar:lib/opencsv-2.3.jar:lib/druid-aws-common-0.9.1.1.jar:lib/jboss-logging-3.1.3.GA.jar:lib/jetty-continuation-9.2.5.v20141112.jar:lib/json-path-2.1.0.jar:lib/curator-recipes-2.10.0.jar:lib/guice-servlet-4.0-beta.jar:lib/druid-api-0.9.1.1.jar:lib/aws-java-sdk-storagegateway-1.10.21.jar:lib/maven-settings-3.1.1.jar:lib/httpcore-4.4.3.jar:lib/log4j-1.2-api-2.5.jar:lib/jetty-servlet-9.2.5.v20141112.jar:lib/log4j-api-2.5.jar:lib/druid-indexing-service-0.9.1.1.jar:lib/javax.inject-1.jar:lib/airline-0.7.jar:lib/server-metrics-0.2.8.jar:lib/commons-dbcp2-2.0.1.jar:lib/jersey-core-1.19.jar:lib/aws-java-sdk-autoscaling-1.10.21.jar:lib/aws-java-sdk-codepipeline-1.10.21.jar:lib/aws-java-sdk-ses-1.10.21.jar:lib/jetty-util-9.2.5.v20141112.jar:lib/aws-java-sdk-lambda-1.10.21.jar:lib/aws-java-sdk-ssm-1.10.21.jar:lib/jetty-proxy-9.2.5.v20141112.jar:lib/jackson-databind-2.4.6.jar:lib/commons-logging-1.1.1.jar:lib/aws-java-sdk-directory-1.10.21.jar:lib/guice-4.0-beta.jar:lib/geoip2-0.4.0.jar:lib/activation-1.1.1.jar:lib/druid-server-0.9.1.1.jar:lib/aws-java-sdk-importexport-1.10.21.jar:lib/commons-pool-1.6.jar:lib/config-magic-0.9.jar:lib/log4j-jul-2.5.jar:lib/commons-cli-1.2.jar:lib/derbynet-10.11.1.1.jar:lib/jersey-guice-1.19.jar:lib/druid-processing-0.9.1.1.jar:lib/jackson-jaxrs-json-provider-2.4.6.jar:lib/aws-java-sdk-route53-1.10.21.jar:lib/aws-java-sdk-glacier-1.10.21.jar:lib/javax.el-3.0.0.jar:lib/aws-java-sdk-simpledb-1.10.21.jar:lib/aws-java-sdk-cloudwatch-1.10.21.jar:lib/jets3t-0.9.4.jar:lib/aws-java-sdk-sns-1.10.21.jar:lib/jackson-jaxrs-base-2.4.6.jar:lib/aws-java-sdk-sts-1.10.21.jar:lib/maven-repository-metadata-3.1.1.jar:lib/aws-java-sdk-cloudhsm-1.10.21.jar:lib/aws-java-sdk-cloudsearch-1.10.21.jar:lib/bytebuffer-collections-0.2.4.jar:lib/http-client-1.0.4.jar:lib/base64-2.3.8.jar:lib/aopalliance-1.0.jar:lib/aws-java-sdk-rds-1.10.21.jar:lib/aether-util-0.9.0.M2.jar:lib/antlr4-runtime-4.5.1.jar:lib/plexus-interpolation-1.19.jar:lib/jline-0.9.94.jar:lib/jersey-servlet-1.19.jar:lib/aws-java-sdk-cloudfront-1.10.21.jar:lib/spymemcached-2.11.7.jar:lib/aws-java-sdk-cloudwatchmetrics-1.10.21.jar:lib/guice-multibindings-4.0-beta.jar:lib/aws-java-sdk-sqs-1.10.21.jar:lib/jetty-server-9.2.5.v20141112.jar:lib/bcprov-jdk15on-1.52.jar:lib/jsr311-api-1.1.1.jar:lib/aws-java-sdk-workspaces-1.10.21.jar:lib/jackson-core-asl-1.9.13.jar:lib/curator-framework-2.10.0.jar:lib/commons-pool2-2.2.jar:lib/aws-java-sdk-core-1.10.21.jar:lib/java-xmlbuilder-1.1.jar:lib/jersey-server-1.19.jar:lib/guava-16.0.1.jar:lib/disruptor-3.3.0.jar:lib/aws-java-sdk-ec2-1.10.21.jar:lib/aws-java-sdk-support-1.10.21.jar:lib/jackson-module-jaxb-annotations-2.4.6.jar:lib/aws-java-sdk-dynamodb-1.10.21.jar:lib/maven-model-builder-3.1.1.jar:lib/validation-api-1.1.0.Final.jar:lib/commons-lang-2.6.jar:lib/aws-java-sdk-cognitoidentity-1.10.21.jar:lib/java-util-0.27.9.jar:lib/aether-connector-file-0.9.0.M2.jar:lib/jackson-mapper-asl-1.9.13.jar:lib/jcl-over-slf4j-1.7.12.jar:lib/httpclient-4.5.1.jar:lib/google-http-client-jackson2-1.15.0-rc.jar:lib/aws-java-sdk-elastictranscoder-1.10.21.jar:lib/aws-java-sdk-iam-1.10.21.jar:lib/rhino-1.7R5.jar:lib/maven-aether-provider-3.1.1.jar:lib/druid-console-0.0.2.jar:lib/commons-io-2.4.jar:lib/RoaringBitmap-0.5.16.jar:lib/aws-java-sdk-redshift-1.10.21.jar:lib/jackson-datatype-joda-2.4.6.jar:lib/compress-lzf-1.0.3.jar:lib/maxminddb-0.2.0.jar:lib/jetty-client-9.2.5.v20141112.jar:lib/javax.el-api-3.0.0.jar:lib/jsr305-2.0.1.jar:lib/aws-java-sdk-elasticbeanstalk-1.10.21.jar:lib/slf4j-api-1.7.12.jar:lib/aether-connector-okhttp-0.0.9.jar:lib/druid-services-0.9.1.1.jar:lib/aws-java-sdk-elasticloadbalancing-1.10.21.jar:lib/aether-impl-0.9.0.M2.jar:lib/derbyclient-10.11.1.1.jar:lib/aws-java-sdk-efs-1.10.21.jar:lib/aws-java-sdk-cloudtrail-1.10.21.jar:lib/jackson-jaxrs-smile-provider-2.4.6.jar:lib/protobuf-java-2.5.0.jar:lib/emitter-0.3.6.jar:lib/jetty-security-9.2.5.v20141112.jar:lib/jdbi-2.63.1.jar:lib/aws-java-sdk-cognitosync-1.10.21.jar:lib/aws-java-sdk-swf-libraries-1.10.21.jar:lib/aws-java-sdk-codedeploy-1.10.21.jar:lib/hibernate-validator-5.1.3.Final.jar:lib/aws-java-sdk-elasticache-1.10.21.jar:lib/aws-java-sdk-devicefarm-1.10.21.jar:lib/lz4-1.3.0.jar:lib/curator-client-2.10.0.jar:lib/zookeeper-3.4.8.jar:lib/maven-settings-builder-3.1.1.jar:lib/aws-java-sdk-simpleworkflow-1.10.21.jar:lib/jetty-http-9.2.5.v20141112.jar:lib/curator-x-discovery-2.10.0.jar:lib/commons-codec-1.7.jar:lib/aws-java-sdk-kms-1.10.21.jar:lib/jackson-annotations-2.4.6.jar:lib/javax.servlet-api-3.1.0.jar:lib/derby-10.11.1.1.jar:lib/aws-java-sdk-config-1.10.21.jar:lib/commons-math3-3.6.1.jar:lib/joda-time-2.8.2.jar:lib/icu4j-4.8.1.jar:lib/irc-api-1.0-0014.jar:lib/aws-java-sdk-emr-1.10.21.jar:lib/aether-api-0.9.0.M2.jar:lib/annotations-2.0.3.jar:lib/jackson-dataformat-smile-2.4.6.jar:lib/aws-java-sdk-cloudformation-1.10.21.jar:lib/jackson-core-2.4.6.jar:lib/aws-java-sdk-directconnect-1.10.21.jar:lib/aws-java-sdk-s3-1.10.21.jar:lib/plexus-utils-3.0.15.jar:lib/extendedset-1.3.9.jar:lib/classmate-1.0.0.jar:lib/aws-java-sdk-opsworks-1.10.21.jar:lib/log4j-core-2.5.jar:lib/aws-java-sdk-logs-1.10.21.jar:lib/jetty-io-9.2.5.v20141112.jar:lib/aws-java-sdk-datapipeline-1.10.21.jar:lib/aws-java-sdk-kinesis-1.10.21.jar:lib/wagon-provider-api-2.4.jar:lib/aether-spi-0.9.0.M2.jar:lib/jackson-datatype-guava-2.4.6.jar:lib/aws-java-sdk-ecs-1.10.21.jar:lib/okhttp-1.0.2.jar:lib/jetty-servlets-9.2.5.v20141112.jar:lib/aws-java-sdk-machinelearning-1.10.21.jar:lib/netty-3.10.4.Final.jar:lib/druid-common-0.9.1.1.jar:lib/aws-java-sdk-1.10.21.jar:lib/log4j-slf4j-impl-2.5.jar
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=var/tmp
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.version=3.13.0-83-generic
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.name=ubuntu
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/ubuntu
2016-08-07T13:09:29,047 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/tmp/druid-0.9.1.1
2016-08-07T13:09:29,048 INFO [main] org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=hadoop02 sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@6fa2448b
2016-08-07T13:09:29,062 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.discovery.ServerDiscoverySelector.start() throws java.lang.Exception] on object[io.druid.curator.discovery.ServerDiscoverySelector@d13baac].
2016-08-07T13:09:29,065 INFO [main-SendThread(hadoop02:2181)] org.apache.zookeeper.ClientCnxn - Opening socket connection to server hadoop02/10.10.70.17:2181. Will not attempt to authenticate using SASL (unknown error)
2016-08-07T13:09:29,071 INFO [main-SendThread(hadoop02:2181)] org.apache.zookeeper.ClientCnxn - Socket connection established to hadoop02/10.10.70.17:2181, initiating session
2016-08-07T13:09:29,078 INFO [main-SendThread(hadoop02:2181)] org.apache.zookeeper.ClientCnxn - Session establishment complete on server hadoop02/10.10.70.17:2181, sessionid = 0x15656ad562a1523, negotiated timeout = 30000
2016-08-07T13:09:29,083 INFO [main-EventThread] org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
2016-08-07T13:09:29,170 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.announcement.Announcer.start()] on object[io.druid.curator.announcement.Announcer@235c997d].
2016-08-07T13:09:29,170 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.discovery.ServerDiscoverySelector.start() throws java.lang.Exception] on object[io.druid.curator.discovery.ServerDiscoverySelector@36bed37a].
2016-08-07T13:09:29,174 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.indexing.worker.executor.ExecutorLifecycle.start() throws java.lang.InterruptedException] on object[io.druid.indexing.worker.executor.ExecutorLifecycle@7f42e06e].
2016-08-07T13:09:29,222 INFO [main] io.druid.indexing.worker.executor.ExecutorLifecycle - Running with task: {
  "type" : "index_hadoop",
  "id" : "index_hadoop_pageviews_2016-08-07T13:09:25.331Z",
  "spec" : {
    "dataSchema" : {
      "dataSource" : "pageviews",
      "parser" : {
        "type" : "string",
        "parseSpec" : {
          "format" : "json",
          "dimensionsSpec" : {
            "dimensions" : [ "url", "user" ]
          },
          "timestampSpec" : {
            "format" : "auto",
            "column" : "time"
          }
        }
      },
      "metricsSpec" : [ {
        "type" : "count",
        "name" : "views"
      }, {
        "type" : "doubleSum",
        "name" : "latencyMs",
        "fieldName" : "latencyMs"
      } ],
      "granularitySpec" : {
        "type" : "uniform",
        "segmentGranularity" : "DAY",
        "queryGranularity" : {
          "type" : "none"
        },
        "intervals" : [ "2016-08-05T00:00:00.000Z/2016-08-06T00:00:00.000Z" ]
      }
    },
    "ioConfig" : {
      "type" : "hadoop",
      "inputSpec" : {
        "type" : "static",
        "paths" : "/tmp/pageviews.json"
      },
      "metadataUpdateSpec" : null,
      "segmentOutputPath" : null
    },
    "tuningConfig" : {
      "type" : "hadoop",
      "workingPath" : null,
      "version" : "2016-08-07T13:09:25.331Z",
      "partitionsSpec" : {
        "type" : "hashed",
        "targetPartitionSize" : 5000000,
        "maxPartitionSize" : 7500000,
        "assumeGrouped" : false,
        "numShards" : -1,
        "partitionDimensions" : [ ]
      },
      "shardSpecs" : { },
      "indexSpec" : {
        "bitmap" : {
          "type" : "concise"
        },
        "dimensionCompression" : null,
        "metricCompression" : null
      },
      "maxRowsInMemory" : 75000,
      "leaveIntermediate" : false,
      "cleanupOnFailure" : true,
      "overwriteFiles" : false,
      "ignoreInvalidRows" : false,
      "jobProperties" : {
        "mapreduce.job.user.classpath.first" : "true"
      },
      "combineText" : false,
      "useCombiner" : false,
      "buildV9Directly" : false,
      "numBackgroundPersistThreads" : 0
    },
    "uniqueId" : "ce644d2573c347cc809275d796b1564e"
  },
  "hadoopDependencyCoordinates" : null,
  "classpathPrefix" : null,
  "context" : null,
  "groupId" : "index_hadoop_pageviews_2016-08-07T13:09:25.331Z",
  "dataSource" : "pageviews",
  "resource" : {
    "availabilityGroup" : "index_hadoop_pageviews_2016-08-07T13:09:25.331Z",
    "requiredCapacity" : 1
  }
}
2016-08-07T13:09:29,224 INFO [main] io.druid.indexing.worker.executor.ExecutorLifecycle - Attempting to lock file[var/tmp/persistent/task/index_hadoop_pageviews_2016-08-07T13:09:25.331Z/lock].
2016-08-07T13:09:29,225 INFO [main] io.druid.indexing.worker.executor.ExecutorLifecycle - Acquired lock file[var/tmp/persistent/task/index_hadoop_pageviews_2016-08-07T13:09:25.331Z/lock] in 1ms.
2016-08-07T13:09:29,228 INFO [main] io.druid.indexing.common.actions.RemoteTaskActionClient - Performing action for task[index_hadoop_pageviews_2016-08-07T13:09:25.331Z]: LockTryAcquireAction{interval=2016-08-05T00:00:00.000Z/2016-08-06T00:00:00.000Z}
2016-08-07T13:09:29,237 INFO [main] io.druid.indexing.common.actions.RemoteTaskActionClient - Submitting action for task[index_hadoop_pageviews_2016-08-07T13:09:25.331Z] to overlord[http://hadoop02:8090/druid/indexer/v1/action]: LockTryAcquireAction{interval=2016-08-05T00:00:00.000Z/2016-08-06T00:00:00.000Z}
2016-08-07T13:09:29,250 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,289 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,289 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,289 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,290 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,290 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,290 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,290 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,291 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,291 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,291 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,291 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,291 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,292 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,292 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,292 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,292 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,292 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,293 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,293 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://hadoop02:8090
2016-08-07T13:09:29,345 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Running task: index_hadoop_pageviews_2016-08-07T13:09:25.331Z
2016-08-07T13:09:29,346 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_hadoop_pageviews_2016-08-07T13:09:25.331Z] location changed to [TaskLocation{host='hadoop02', port=8100}].
2016-08-07T13:09:29,347 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_hadoop_pageviews_2016-08-07T13:09:25.331Z] status changed to [RUNNING].
2016-08-07T13:09:29,349 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/xercesImpl-2.9.1.jar]
2016-08-07T13:09:29,350 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/api-asn1-api-1.0.0-M20.jar]
2016-08-07T13:09:29,350 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/stax-api-1.0-2.jar]
2016-08-07T13:09:29,350 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-yarn-common-2.6.0.jar]
2016-08-07T13:09:29,350 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-compress-1.4.1.jar]
2016-08-07T13:09:29,350 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/slf4j-api-1.7.5.jar]
2016-08-07T13:09:29,350 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-hdfs-2.6.0.jar]
2016-08-07T13:09:29,351 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/httpcore-4.2.5.jar]
2016-08-07T13:09:29,351 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-math3-3.1.1.jar]
2016-08-07T13:09:29,351 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/curator-recipes-2.6.0.jar]
2016-08-07T13:09:29,351 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/xz-1.0.jar]
2016-08-07T13:09:29,351 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/slf4j-log4j12-1.7.5.jar]
2016-08-07T13:09:29,351 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-beanutils-1.7.0.jar]
2016-08-07T13:09:29,351 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-cli-1.2.jar]
2016-08-07T13:09:29,352 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/curator-client-2.6.0.jar]
2016-08-07T13:09:29,352 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/servlet-api-2.5.jar]
2016-08-07T13:09:29,352 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/apacheds-kerberos-codec-2.0.0-M15.jar]
2016-08-07T13:09:29,352 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/activation-1.1.jar]
2016-08-07T13:09:29,352 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-mapreduce-client-app-2.6.0.jar]
2016-08-07T13:09:29,352 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-net-3.1.jar]
2016-08-07T13:09:29,353 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jersey-client-1.9.jar]
2016-08-07T13:09:29,353 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-configuration-1.6.jar]
2016-08-07T13:09:29,353 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/curator-framework-2.6.0.jar]
2016-08-07T13:09:29,353 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/httpclient-4.2.5.jar]
2016-08-07T13:09:29,353 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/htrace-core-3.0.4.jar]
2016-08-07T13:09:29,353 INFO [main] org.eclipse.jetty.server.Server - jetty-9.2.5.v20141112
2016-08-07T13:09:29,353 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-digester-1.8.jar]
2016-08-07T13:09:29,354 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-yarn-client-2.6.0.jar]
2016-08-07T13:09:29,354 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jaxb-api-2.2.2.jar]
2016-08-07T13:09:29,354 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jackson-core-asl-1.9.13.jar]
2016-08-07T13:09:29,354 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-common-2.6.0.jar]
2016-08-07T13:09:29,355 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-httpclient-3.1.jar]
2016-08-07T13:09:29,355 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-mapreduce-client-shuffle-2.6.0.jar]
2016-08-07T13:09:29,355 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-mapreduce-client-core-2.6.0.jar]
2016-08-07T13:09:29,355 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jsr305-1.3.9.jar]
2016-08-07T13:09:29,355 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/xmlenc-0.52.jar]
2016-08-07T13:09:29,355 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-lang-2.6.jar]
2016-08-07T13:09:29,355 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jackson-mapper-asl-1.9.13.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/guava-11.0.2.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-io-2.4.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/xml-apis-1.3.04.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-yarn-api-2.6.0.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/netty-3.6.2.Final.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/protobuf-java-2.5.0.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-annotations-2.6.0.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-mapreduce-client-common-2.6.0.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/leveldbjni-all-1.8.jar]
2016-08-07T13:09:29,356 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/avro-1.7.4.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/gson-2.2.4.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-yarn-server-common-2.6.0.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-codec-1.4.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-logging-1.1.3.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-client-2.6.0.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/api-util-1.0.0-M20.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-auth-2.6.0.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/hadoop-mapreduce-client-jobclient-2.6.0.jar]
2016-08-07T13:09:29,357 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/apacheds-i18n-2.0.0-M15.jar]
2016-08-07T13:09:29,358 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/paranamer-2.3.jar]
2016-08-07T13:09:29,358 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-beanutils-core-1.8.0.jar]
2016-08-07T13:09:29,358 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jackson-jaxrs-1.9.13.jar]
2016-08-07T13:09:29,358 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/log4j-1.2.17.jar]
2016-08-07T13:09:29,358 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/commons-collections-3.2.1.jar]
2016-08-07T13:09:29,358 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/snappy-java-1.0.4.1.jar]
2016-08-07T13:09:29,358 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jackson-xc-1.9.13.jar]
2016-08-07T13:09:29,359 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jersey-core-1.9.jar]
2016-08-07T13:09:29,360 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/zookeeper-3.4.6.jar]
2016-08-07T13:09:29,360 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/jetty-util-6.1.26.jar]
2016-08-07T13:09:29,360 INFO [task-runner-0-priority-0] io.druid.indexing.common.task.HadoopTask - Hadoop Container Druid Classpath is set to [file:/tmp/druid-0.9.1.1/conf/druid/_common/:file:/tmp/druid-0.9.1.1/conf/druid/overlord/:file:/tmp/druid-0.9.1.1/lib/druid-indexing-hadoop-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/lib/mapdb-1.0.8.jar:file:/tmp/druid-0.9.1.1/lib/maven-model-3.1.1.jar:file:/tmp/druid-0.9.1.1/lib/tesla-aether-0.0.5.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-codecommit-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/opencsv-2.3.jar:file:/tmp/druid-0.9.1.1/lib/druid-aws-common-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/lib/jboss-logging-3.1.3.GA.jar:file:/tmp/druid-0.9.1.1/lib/jetty-continuation-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/json-path-2.1.0.jar:file:/tmp/druid-0.9.1.1/lib/curator-recipes-2.10.0.jar:file:/tmp/druid-0.9.1.1/lib/guice-servlet-4.0-beta.jar:file:/tmp/druid-0.9.1.1/lib/druid-api-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-storagegateway-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/maven-settings-3.1.1.jar:file:/tmp/druid-0.9.1.1/lib/httpcore-4.4.3.jar:file:/tmp/druid-0.9.1.1/lib/log4j-1.2-api-2.5.jar:file:/tmp/druid-0.9.1.1/lib/jetty-servlet-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/log4j-api-2.5.jar:file:/tmp/druid-0.9.1.1/lib/druid-indexing-service-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/lib/javax.inject-1.jar:file:/tmp/druid-0.9.1.1/lib/airline-0.7.jar:file:/tmp/druid-0.9.1.1/lib/server-metrics-0.2.8.jar:file:/tmp/druid-0.9.1.1/lib/commons-dbcp2-2.0.1.jar:file:/tmp/druid-0.9.1.1/lib/jersey-core-1.19.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-autoscaling-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-codepipeline-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-ses-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jetty-util-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-lambda-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-ssm-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jetty-proxy-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/jackson-databind-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/commons-logging-1.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-directory-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/guice-4.0-beta.jar:file:/tmp/druid-0.9.1.1/lib/geoip2-0.4.0.jar:file:/tmp/druid-0.9.1.1/lib/activation-1.1.1.jar:file:/tmp/druid-0.9.1.1/lib/druid-server-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-importexport-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/commons-pool-1.6.jar:file:/tmp/druid-0.9.1.1/lib/config-magic-0.9.jar:file:/tmp/druid-0.9.1.1/lib/log4j-jul-2.5.jar:file:/tmp/druid-0.9.1.1/lib/commons-cli-1.2.jar:file:/tmp/druid-0.9.1.1/lib/derbynet-10.11.1.1.jar:file:/tmp/druid-0.9.1.1/lib/jersey-guice-1.19.jar:file:/tmp/druid-0.9.1.1/lib/druid-processing-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/lib/jackson-jaxrs-json-provider-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-route53-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-glacier-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/javax.el-3.0.0.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-simpledb-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cloudwatch-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jets3t-0.9.4.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-sns-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jackson-jaxrs-base-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-sts-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/maven-repository-metadata-3.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cloudhsm-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cloudsearch-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/bytebuffer-collections-0.2.4.jar:file:/tmp/druid-0.9.1.1/lib/http-client-1.0.4.jar:file:/tmp/druid-0.9.1.1/lib/base64-2.3.8.jar:file:/tmp/druid-0.9.1.1/lib/aopalliance-1.0.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-rds-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aether-util-0.9.0.M2.jar:file:/tmp/druid-0.9.1.1/lib/antlr4-runtime-4.5.1.jar:file:/tmp/druid-0.9.1.1/lib/plexus-interpolation-1.19.jar:file:/tmp/druid-0.9.1.1/lib/jline-0.9.94.jar:file:/tmp/druid-0.9.1.1/lib/jersey-servlet-1.19.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cloudfront-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/spymemcached-2.11.7.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cloudwatchmetrics-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/guice-multibindings-4.0-beta.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-sqs-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jetty-server-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/bcprov-jdk15on-1.52.jar:file:/tmp/druid-0.9.1.1/lib/jsr311-api-1.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-workspaces-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jackson-core-asl-1.9.13.jar:file:/tmp/druid-0.9.1.1/lib/curator-framework-2.10.0.jar:file:/tmp/druid-0.9.1.1/lib/commons-pool2-2.2.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-core-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/java-xmlbuilder-1.1.jar:file:/tmp/druid-0.9.1.1/lib/jersey-server-1.19.jar:file:/tmp/druid-0.9.1.1/lib/guava-16.0.1.jar:file:/tmp/druid-0.9.1.1/lib/disruptor-3.3.0.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-ec2-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-support-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jackson-module-jaxb-annotations-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-dynamodb-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/maven-model-builder-3.1.1.jar:file:/tmp/druid-0.9.1.1/lib/validation-api-1.1.0.Final.jar:file:/tmp/druid-0.9.1.1/lib/commons-lang-2.6.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cognitoidentity-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/java-util-0.27.9.jar:file:/tmp/druid-0.9.1.1/lib/aether-connector-file-0.9.0.M2.jar:file:/tmp/druid-0.9.1.1/lib/jackson-mapper-asl-1.9.13.jar:file:/tmp/druid-0.9.1.1/lib/jcl-over-slf4j-1.7.12.jar:file:/tmp/druid-0.9.1.1/lib/httpclient-4.5.1.jar:file:/tmp/druid-0.9.1.1/lib/google-http-client-jackson2-1.15.0-rc.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-elastictranscoder-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-iam-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/rhino-1.7R5.jar:file:/tmp/druid-0.9.1.1/lib/maven-aether-provider-3.1.1.jar:file:/tmp/druid-0.9.1.1/lib/druid-console-0.0.2.jar:file:/tmp/druid-0.9.1.1/lib/commons-io-2.4.jar:file:/tmp/druid-0.9.1.1/lib/RoaringBitmap-0.5.16.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-redshift-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jackson-datatype-joda-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/compress-lzf-1.0.3.jar:file:/tmp/druid-0.9.1.1/lib/maxminddb-0.2.0.jar:file:/tmp/druid-0.9.1.1/lib/jetty-client-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/javax.el-api-3.0.0.jar:file:/tmp/druid-0.9.1.1/lib/jsr305-2.0.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-elasticbeanstalk-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/slf4j-api-1.7.12.jar:file:/tmp/druid-0.9.1.1/lib/aether-connector-okhttp-0.0.9.jar:file:/tmp/druid-0.9.1.1/lib/druid-services-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-elasticloadbalancing-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aether-impl-0.9.0.M2.jar:file:/tmp/druid-0.9.1.1/lib/derbyclient-10.11.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-efs-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cloudtrail-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jackson-jaxrs-smile-provider-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/protobuf-java-2.5.0.jar:file:/tmp/druid-0.9.1.1/lib/emitter-0.3.6.jar:file:/tmp/druid-0.9.1.1/lib/jetty-security-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/jdbi-2.63.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cognitosync-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-swf-libraries-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-codedeploy-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/hibernate-validator-5.1.3.Final.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-elasticache-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-devicefarm-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/lz4-1.3.0.jar:file:/tmp/druid-0.9.1.1/lib/curator-client-2.10.0.jar:file:/tmp/druid-0.9.1.1/lib/zookeeper-3.4.8.jar:file:/tmp/druid-0.9.1.1/lib/maven-settings-builder-3.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-simpleworkflow-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jetty-http-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/curator-x-discovery-2.10.0.jar:file:/tmp/druid-0.9.1.1/lib/commons-codec-1.7.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-kms-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jackson-annotations-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/javax.servlet-api-3.1.0.jar:file:/tmp/druid-0.9.1.1/lib/derby-10.11.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-config-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/commons-math3-3.6.1.jar:file:/tmp/druid-0.9.1.1/lib/joda-time-2.8.2.jar:file:/tmp/druid-0.9.1.1/lib/icu4j-4.8.1.jar:file:/tmp/druid-0.9.1.1/lib/irc-api-1.0-0014.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-emr-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aether-api-0.9.0.M2.jar:file:/tmp/druid-0.9.1.1/lib/annotations-2.0.3.jar:file:/tmp/druid-0.9.1.1/lib/jackson-dataformat-smile-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-cloudformation-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jackson-core-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-directconnect-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-s3-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/plexus-utils-3.0.15.jar:file:/tmp/druid-0.9.1.1/lib/extendedset-1.3.9.jar:file:/tmp/druid-0.9.1.1/lib/classmate-1.0.0.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-opsworks-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/log4j-core-2.5.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-logs-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/jetty-io-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-datapipeline-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-kinesis-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/wagon-provider-api-2.4.jar:file:/tmp/druid-0.9.1.1/lib/aether-spi-0.9.0.M2.jar:file:/tmp/druid-0.9.1.1/lib/jackson-datatype-guava-2.4.6.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-ecs-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/okhttp-1.0.2.jar:file:/tmp/druid-0.9.1.1/lib/jetty-servlets-9.2.5.v20141112.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-machinelearning-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/netty-3.10.4.Final.jar:file:/tmp/druid-0.9.1.1/lib/druid-common-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/lib/aws-java-sdk-1.10.21.jar:file:/tmp/druid-0.9.1.1/lib/log4j-slf4j-impl-2.5.jar:file:/tmp/druid-0.9.1.1/extensions/druid-histogram/druid-histogram-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/extensions/mysql-metadata-storage/mysql-connector-java-5.1.38.jar:file:/tmp/druid-0.9.1.1/extensions/mysql-metadata-storage/mysql-metadata-storage-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xercesImpl-2.9.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/api-asn1-api-1.0.0-M20.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/stax-api-1.0-2.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-common-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-compress-1.4.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-recipes-2.10.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-hdfs-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xz-1.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/druid-hdfs-storage-0.9.1.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-beanutils-1.7.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/servlet-api-2.5.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/apacheds-kerberos-codec-2.0.0-M15.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-app-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-net-3.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jersey-client-1.9.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-configuration-1.6.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/htrace-core-3.0.4.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-digester-1.8.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-client-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jaxb-api-2.2.2.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-framework-2.10.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-common-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-shuffle-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/guava-16.0.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-core-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xmlenc-0.52.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xml-apis-1.3.04.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-api-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/netty-3.6.2.Final.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-common-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/leveldbjni-all-1.8.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/avro-1.7.4.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-client-2.10.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/gson-2.2.4.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-server-common-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-client-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/api-util-1.0.0-M20.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-math3-3.6.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-auth-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-jobclient-2.6.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/apacheds-i18n-2.0.0-M15.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/paranamer-2.3.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-beanutils-core-1.8.0.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jackson-jaxrs-1.9.13.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-collections-3.2.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/snappy-java-1.0.4.1.jar:file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jackson-xc-1.9.13.jar]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/druid-0.9.1.1/lib/log4j-slf4j-impl-2.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/druid-0.9.1.1/hadoop-dependencies/hadoop-client/2.6.0/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider as a provider class
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering com.fasterxml.jackson.jaxrs.smile.JacksonSmileProvider as a provider class
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering io.druid.server.initialization.jetty.CustomExceptionMapper as a provider class
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering io.druid.server.StatusResource as a root resource class
Aug 07, 2016 1:09:29 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.server.initialization.jetty.CustomExceptionMapper to GuiceManagedComponentProvider with the scope "Singleton"
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider to GuiceManagedComponentProvider with the scope "Singleton"
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding com.fasterxml.jackson.jaxrs.smile.JacksonSmileProvider to GuiceManagedComponentProvider with the scope "Singleton"
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.server.QueryResource to GuiceInstantiatedComponentProvider
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.segment.realtime.firehose.ChatHandlerResource to GuiceInstantiatedComponentProvider
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.query.lookup.LookupListeningResource to GuiceInstantiatedComponentProvider
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.query.lookup.LookupIntrospectionResource to GuiceInstantiatedComponentProvider
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.server.http.security.StateResourceFilter to GuiceInstantiatedComponentProvider
Aug 07, 2016 1:09:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.server.StatusResource to GuiceManagedComponentProvider with the scope "Undefined"
2016-08-07T13:09:29,859 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@65600fb3{/,null,AVAILABLE}
2016-08-07T13:09:29,868 INFO [main] org.eclipse.jetty.server.ServerConnector - Started ServerConnector@5b066c33{HTTP/1.1}{0.0.0.0:8100}
2016-08-07T13:09:29,869 INFO [main] org.eclipse.jetty.server.Server - Started @4520ms
2016-08-07T13:09:29,870 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.query.lookup.LookupReferencesManager.start()] on object[io.druid.query.lookup.LookupReferencesManager@62ea8931].
2016-08-07T13:09:29,870 INFO [main] io.druid.query.lookup.LookupReferencesManager - Started lookup factory references manager
2016-08-07T13:09:29,870 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.server.listener.announcer.ListenerResourceAnnouncer.start()] on object[io.druid.query.lookup.LookupResourceListenerAnnouncer@47fb7ec9].
2016-08-07T13:09:29,886 INFO [main] io.druid.server.listener.announcer.ListenerResourceAnnouncer - Announcing start time on [/druid/listeners/lookups/__default/hadoop02:8100]
2016-08-07T13:09:29,886 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.server.coordination.AbstractDataSegmentAnnouncer.start()] on object[io.druid.server.coordination.BatchDataSegmentAnnouncer@7cf8f45a].
2016-08-07T13:09:29,886 INFO [main] io.druid.server.coordination.AbstractDataSegmentAnnouncer - Announcing self[DruidServerMetadata{name='hadoop02:8100', host='hadoop02:8100', maxSize=0, tier='_default_tier', type='indexer-executor', priority='0'}] at [/druid/announcements/hadoop02:8100]
2016-08-07T13:09:29,976 INFO [task-runner-0-priority-0] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties
2016-08-07T13:09:29,977 INFO [task-runner-0-priority-0] io.druid.guice.PropertiesModule - Loading properties from runtime.properties
2016-08-07T13:09:30,009 INFO [task-runner-0-priority-0] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final
2016-08-07T13:09:30,710 INFO [task-runner-0-priority-0] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, directory='extensions', hadoopDependenciesDir='hadoop-dependencies', hadoopContainerDruidClasspath='null', loadList=[druid-histogram, mysql-metadata-storage, druid-hdfs-storage]}]
2016-08-07T13:09:30,716 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - Adding classpath extension module [io.druid.query.aggregation.histogram.ApproximateHistogramDruidModule] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:30,717 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - Adding classpath extension module [io.druid.metadata.storage.mysql.MySQLMetadataStorageModule] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:30,718 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - Adding classpath extension module [io.druid.storage.hdfs.HdfsStorageDruidModule] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:30,718 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - Loading extension [druid-histogram] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:30,727 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-histogram/druid-histogram-0.9.1.1.jar]
2016-08-07T13:09:30,729 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - Loading extension [mysql-metadata-storage] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:30,729 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/mysql-metadata-storage/mysql-connector-java-5.1.38.jar]
2016-08-07T13:09:30,729 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/mysql-metadata-storage/mysql-metadata-storage-0.9.1.1.jar]
2016-08-07T13:09:30,730 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - Loading extension [druid-hdfs-storage] for class [io.druid.initialization.DruidModule]
2016-08-07T13:09:30,730 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xercesImpl-2.9.1.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/api-asn1-api-1.0.0-M20.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/stax-api-1.0-2.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-common-2.6.0.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-compress-1.4.1.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-recipes-2.10.0.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-hdfs-2.6.0.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xz-1.0.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/druid-hdfs-storage-0.9.1.1.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-beanutils-1.7.0.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/servlet-api-2.5.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/apacheds-kerberos-codec-2.0.0-M15.jar]
2016-08-07T13:09:30,731 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-app-2.6.0.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-net-3.1.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jersey-client-1.9.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-configuration-1.6.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/htrace-core-3.0.4.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-digester-1.8.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-client-2.6.0.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jaxb-api-2.2.2.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-framework-2.10.0.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-common-2.6.0.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-shuffle-2.6.0.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/guava-16.0.1.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-core-2.6.0.jar]
2016-08-07T13:09:30,732 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xmlenc-0.52.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/xml-apis-1.3.04.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-api-2.6.0.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/netty-3.6.2.Final.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-common-2.6.0.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/leveldbjni-all-1.8.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/avro-1.7.4.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/curator-client-2.10.0.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/gson-2.2.4.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-yarn-server-common-2.6.0.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-client-2.6.0.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/api-util-1.0.0-M20.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-math3-3.6.1.jar]
2016-08-07T13:09:30,733 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-auth-2.6.0.jar]
2016-08-07T13:09:30,734 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/hadoop-mapreduce-client-jobclient-2.6.0.jar]
2016-08-07T13:09:30,734 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/apacheds-i18n-2.0.0-M15.jar]
2016-08-07T13:09:30,734 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/paranamer-2.3.jar]
2016-08-07T13:09:30,734 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-beanutils-core-1.8.0.jar]
2016-08-07T13:09:30,734 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jackson-jaxrs-1.9.13.jar]
2016-08-07T13:09:30,734 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/commons-collections-3.2.1.jar]
2016-08-07T13:09:30,734 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/snappy-java-1.0.4.1.jar]
2016-08-07T13:09:30,734 INFO [task-runner-0-priority-0] io.druid.initialization.Initialization - added URL[file:/tmp/druid-0.9.1.1/extensions/druid-hdfs-storage/jackson-xc-1.9.13.jar]
2016-08-07T13:09:31,531 WARN [task-runner-0-priority-0] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-08-07T13:09:31,757 INFO [task-runner-0-priority-0] io.druid.guice.JsonConfigurator - Loaded class[class com.metamx.emitter.core.LoggingEmitterConfig] from props[druid.emitter.logging.] as [LoggingEmitterConfig{loggerClass='com.metamx.emitter.core.LoggingEmitter', logLevel='info'}]
2016-08-07T13:09:31,816 INFO [task-runner-0-priority-0] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@7b88c1c6]
2016-08-07T13:09:31,823 INFO [task-runner-0-priority-0] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[]}]
2016-08-07T13:09:31,828 INFO [task-runner-0-priority-0] io.druid.server.metrics.MetricsModule - Adding monitor[io.druid.query.ExecutorServiceMonitor@400c4528]
2016-08-07T13:09:31,828 INFO [task-runner-0-priority-0] io.druid.server.metrics.MetricsModule - Adding monitor[io.druid.server.initialization.jetty.JettyServerModule$JettyMonitor@a50f41d]
2016-08-07T13:09:31,832 INFO [task-runner-0-priority-0] org.skife.config.ConfigurationObjectFactory - Assigning value [53687] for [druid.processing.buffer.sizeBytes] on [io.druid.query.DruidProcessingConfig#intermediateComputeSizeBytes()]
2016-08-07T13:09:31,833 INFO [task-runner-0-priority-0] org.skife.config.ConfigurationObjectFactory - Assigning value [1] for [druid.processing.numThreads] on [io.druid.query.DruidProcessingConfig#getNumThreads()]
2016-08-07T13:09:31,833 INFO [task-runner-0-priority-0] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.computation.buffer.poolCacheMaxCount, ${base_path}.buffer.poolCacheMaxCount] on [io.druid.query.DruidProcessingConfig#poolCacheMaxCount()]
2016-08-07T13:09:31,834 INFO [task-runner-0-priority-0] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.columnCache.sizeBytes] on [io.druid.query.DruidProcessingConfig#columnCacheSizeBytes()]
2016-08-07T13:09:31,835 INFO [task-runner-0-priority-0] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.fifo] on [io.druid.query.DruidProcessingConfig#isFifo()]
2016-08-07T13:09:31,835 INFO [task-runner-0-priority-0] org.skife.config.ConfigurationObjectFactory - Assigning default value [processing-%s] for [${base_path}.formatString] on [com.metamx.common.concurrent.ExecutorServiceConfig#getFormatString()]
2016-08-07T13:09:32,040 INFO [task-runner-0-priority-0] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties
2016-08-07T13:09:32,041 INFO [task-runner-0-priority-0] io.druid.guice.PropertiesModule - Loading properties from runtime.properties
2016-08-07T13:09:32,050 INFO [task-runner-0-priority-0] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, directory='extensions', hadoopDependenciesDir='hadoop-dependencies', hadoopContainerDruidClasspath='null', loadList=[druid-histogram, mysql-metadata-storage, druid-hdfs-storage]}]
2016-08-07T13:09:32,050 INFO [task-runner-0-priority-0] io.druid.indexing.common.task.HadoopIndexTask - Starting a hadoop determine configuration job...
2016-08-07T13:09:32,082 INFO [task-runner-0-priority-0] io.druid.indexer.path.StaticPathSpec - Adding paths[/tmp/pageviews.json]
2016-08-07T13:09:32,486 INFO [task-runner-0-priority-0] io.druid.indexer.path.StaticPathSpec - Adding paths[/tmp/pageviews.json]
2016-08-07T13:09:32,580 INFO [task-runner-0-priority-0] org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at hadoop00/10.10.213.126:8032
2016-08-07T13:09:32,826 WARN [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobSubmitter - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2016-08-07T13:09:32,832 WARN [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobSubmitter - No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2016-08-07T13:09:33,056 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2016-08-07T13:09:33,259 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
2016-08-07T13:09:33,351 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1470197360159_0277
2016-08-07T13:09:33,467 INFO [task-runner-0-priority-0] org.apache.hadoop.mapred.YARNRunner - Job jar is not present. Not adding any jar to the list of resources.
2016-08-07T13:09:33,781 INFO [task-runner-0-priority-0] org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1470197360159_0277
2016-08-07T13:09:33,814 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.Job - The url to track the job: http://hadoop00:8088/proxy/application_1470197360159_0277/
2016-08-07T13:09:33,814 INFO [task-runner-0-priority-0] io.druid.indexer.DetermineHashedPartitionsJob - Job pageviews-determine_partitions_hashed-Optional.of([2016-08-05T00:00:00.000Z/2016-08-06T00:00:00.000Z]) submitted, status available at: http://hadoop00:8088/proxy/application_1470197360159_0277/
2016-08-07T13:09:33,815 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.Job - Running job: job_1470197360159_0277

David Lim

unread,
Aug 8, 2016, 12:16:30 PM8/8/16
to Druid User
Hi Yi,

Have you looked into the Hadoop job logs to see if there are any clues there as to what's happening?

Yi

unread,
Aug 10, 2016, 6:05:44 AM8/10/16
to Druid User
Hi David,

Here is job logs:
va.lang.ArithmeticException: / by zero
	at org.apache.hadoop.mapreduce.v2.app.rm.ResourceCalculatorUtils.computeAvailableContainers(ResourceCalculatorUtils.java:38) ~[hadoop-mapreduce-client-app-2.6.0.jar:?]
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.preemptReducesIfNeeded(RMContainerAllocator.java:461) ~[hadoop-mapreduce-client-app-2.6.0.jar:?]
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:258) ~[hadoop-mapreduce-client-app-2.6.0.jar:?]
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$1.run(RMCommunicator.java:282) [hadoop-mapreduce-client-app-2.6.0.jar:?]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
2016-08-10T17:54:03,221 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator - ERROR IN CONTACTING RM. 
java.lang.ArithmeticException: / by zero
	at org.apache.hadoop.mapreduce.v2.app.rm.ResourceCalculatorUtils.computeAvailableContainers(ResourceCalculatorUtils.java:38) ~[hadoop-mapreduce-client-app-2.6.0.jar:?]
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.preemptReducesIfNeeded(RMContainerAllocator.java:461) ~[hadoop-mapreduce-client-app-2.6.0.jar:?]
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:258) ~[hadoop-mapreduce-client-app-2.6.0.jar:?]
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$1.run(RMCommunicator.java:282) [hadoop-mapreduce-client-app-2.6.0.jar:?]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
2016-08-10T17:54:04,223 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator - ERROR IN CONTACTING RM.


Maybe there are issues with the druid I rebuild? I just changed Hadoop version in pom.xml
...

Nishant Bangarwa

unread,
Aug 10, 2016, 6:22:17 AM8/10/16
to Druid User
Hi, 
Looks like a hadoop config issue, 
have you set the mapreduce.map.cpu.vcores or mapreduce.reduce.cpu.vcores to 0 in your hadoop configs ? 
If thats the case try changing it to non-zero value.  

Cheers, 
Nishant

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/34fdb839-f4ce-438e-bdc8-3d4944530669%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Yi

unread,
Aug 11, 2016, 12:00:39 AM8/11/16
to Druid User
Hi, 

here is my mapred-site.xml which copied from /etc/hadoop/conf/

<?xml version="1.0" encoding="UTF-8"?>

<!--Autogenerated by Cloudera Manager-->
<configuration>
  <property>
    <name>mapreduce.job.split.metainfo.maxsize</name>
    <value>10000000</value>
  </property>
  <property>
    <name>mapreduce.job.counters.max</name>
    <value>120</value>
  </property>
  <property>
    <name>mapreduce.output.fileoutputformat.compress</name>
    <value>false</value>
  </property>
  <property>
    <name>mapreduce.output.fileoutputformat.compress.type</name>
    <value>BLOCK</value>
  </property>
  <property>
    <name>mapreduce.output.fileoutputformat.compress.codec</name>
    <value>org.apache.hadoop.io.compress.DefaultCodec</value>
  </property>
  <property>
    <name>mapreduce.map.output.compress.codec</name>
    <value>org.apache.hadoop.io.compress.SnappyCodec</value>
  </property>
  <property>
    <name>mapreduce.map.output.compress</name>
    <value>true</value>
  </property>
  <property>
    <name>zlib.compress.level</name>
    <value>DEFAULT_COMPRESSION</value>
  </property>
  <property>
    <name>mapreduce.task.io.sort.factor</name>
    <value>64</value>
  </property>
  <property>
    <name>mapreduce.map.sort.spill.percent</name>
    <value>0.8</value>
  </property>
  <property>
    <name>mapreduce.reduce.shuffle.parallelcopies</name>
    <value>10</value>
  </property>
  <property>
    <name>mapreduce.task.timeout</name>
    <value>600000</value>
  </property>
  <property>
    <name>mapreduce.client.submit.file.replication</name>
    <value>1</value>
  </property>
  <property>
    <name>mapreduce.job.reduces</name>
    <value>12</value>
  </property>
  <property>
    <name>mapreduce.task.io.sort.mb</name>
    <value>256</value>
  </property>
  <property>
    <name>mapreduce.map.speculative</name>
    <value>false</value>
  </property>
  <property>
    <name>mapreduce.reduce.speculative</name>
    <value>false</value>
  </property>
  <property>
    <name>mapreduce.job.reduce.slowstart.completedmaps</name>
    <value>0.8</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.address</name>
    <value>hadoop02:10020</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>hadoop02:19888</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.webapp.https.address</name>
    <value>hadoop02:19890</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.admin.address</name>
    <value>hadoop02:10033</value>
  </property>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
  <property>
    <name>yarn.app.mapreduce.am.staging-dir</name>
    <value>/user</value>
  </property>
  <property>
    <name>mapreduce.am.max-attempts</name>
    <value>2</value>
  </property>
  <property>
    <name>yarn.app.mapreduce.am.resource.mb</name>
    <value>1024</value>
  </property>
  <property>
    <name>yarn.app.mapreduce.am.resource.cpu-vcores</name>
    <value>1</value>
  </property>
  <property>
    <name>mapreduce.job.ubertask.enable</name>
    <value>false</value>
  </property>
  <property>
    <name>yarn.app.mapreduce.am.command-opts</name>
    <value>-Djava.net.preferIPv4Stack=true -Xmx825955249</value>
  </property>
  <property>
    <name>mapreduce.map.java.opts</name>
    <value>-Djava.net.preferIPv4Stack=true -Duser.timezone=UTC -Dfile.encoding=UTF-8</value>
  </property>
  <property>
    <name>mapreduce.reduce.java.opts</name>
    <value>-Djava.net.preferIPv4Stack=true -Duser.timezone=UTC -Dfile.encoding=UTF-8</value>
  </property>
  <property>
    <name>yarn.app.mapreduce.am.admin.user.env</name>
    <value>LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native:$JAVA_LIBRARY_PATH</value>
  </property>
  <property>
    <name>mapreduce.map.memory.mb</name>
    <value>0</value>
  </property>
  <property>
    <name>mapreduce.map.cpu.vcores</name>
    <value>1</value>
  </property>
  <property>
    <name>mapreduce.reduce.memory.mb</name>
    <value>0</value>
  </property>
  <property>
    <name>mapreduce.reduce.cpu.vcores</name>
    <value>1</value>
  </property>
  <property>
    <name>mapreduce.job.heap.memory-mb.ratio</name>
    <value>0.8</value>
  </property>
  <property>
    <name>mapreduce.application.classpath</name>
    <value>$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,$MR2_CLASSPATH</value>
  </property>
  <property>
    <name>mapreduce.admin.user.env</name>
    <value>LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native:$JAVA_LIBRARY_PATH</value>
  </property>
  <property>
    <name>mapreduce.shuffle.max.connections</name>
    <value>80</value>
  </property>
</configuration>

Fangjin Yang

unread,
Aug 15, 2016, 8:15:35 PM8/15/16
to Druid User
Hi Yi, that is quite strange. How are you including mapred-site.xml in your classpath?
...

Yi

unread,
Aug 22, 2016, 4:26:01 AM8/22/16
to Druid User
Hi Fangjin,

I place  (core-site.xml, hdfs-site.xml, yarn-site.xml, mapred-site.xml) into conf/druid/_common/.
...

Jonathan Wei

unread,
Aug 22, 2016, 6:10:07 PM8/22/16
to druid...@googlegroups.com
Looking at the hadoop ResourceCalculatorUtils code, a 0 value for memory parameters can also trigger that divide by zero error:


Can you try setting:

mapreduce.map.memory.mb
mapreduce.reduce.memory.mb

to non-zero values?

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+unsubscribe@googlegroups.com.

To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/6bcbc88a-5b10-469b-b7a7-722442abbdd5%40googlegroups.com.

Yi

unread,
Aug 23, 2016, 5:31:29 AM8/23/16
to Druid User
Thaks Jonathan ,  druid works perfect after I changed  mapreduce.map.memory.mb/mapreduce.reduce.memory.mb 
...
Reply all
Reply to author
Forward
0 new messages