Druid default segment id (timestamp) format results in invalid DFS filename
205 views
Skip to first unread message
Michael
May 13, 2014, 1:46:25 AM5/13/14
to druid-de...@googlegroups.com
Hi,
I was trying to do a hadoop indexing task, and it seemed finished its MR work and most of the processing, but near the end it threw an exception about my path name for hdfs deepstore - since part of the path name contains Druid's standard "hh:mm:ss" style id or interval representations (DFS does not allow ":", I believe?)
Is there anyway I can config customized id/interval representation so I can avoid ":"?
Thanks.
Michael
I was trying to do a hadoop indexing task, and it seemed finished its MR work and most of the processing, but near the end it threw an exception about my path name for hdfs deepstore - since part of the path name contains Druid's standard "hh:mm:ss" style id or interval representations (DFS does not allow ":", I believe?)
Is there anyway I can config customized id/interval representation so I can avoid ":"?
Thanks.
Michael
Nishant Bangarwa
May 13, 2014, 4:05:01 AM5/13/14
to druid-de...@googlegroups.com
Hi Michael,
druid already handles replacing ":" with "_" while calculating path for deep storage for hdfs.
Its handled here - https://github.com/druid-io/druid-api/blob/master/src/main/java/io/druid/segment/loading/DataSegmentPusherUtil.java
I wonder if its failing due to any other configuration issue, can you share the logs containing the exception for more context ?
Michael
May 13, 2014, 1:42:39 PM5/13/14
to druid-de...@googlegroups.com
Entire log msg here:
--------------------------------------
--------------------------------------
SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/y/libexec/mdi_druid/lib/slf4j-simple-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/y/libexec/mdi_druid/lib/slf4j-log4j12-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory] 231 [main] INFO io.druid.server.initialization.PropertiesModule - Loading properties from runtime.properties 2014-05-13 01:17:02,481 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.0.1.Final 1099 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[], localRepository='/home/y/tmp/.m2/repository', remoteRepositories=[]}] 1444 [main] INFO io.druid.initialization.Initialization - Adding local module[class io.druid.storage.hdfs.HdfsStorageDruidModule] 1445 [main] INFO io.druid.initialization.Initialization - Adding local module[class io.druid.query.aggregation.HyperloglogDruidModule] 1447 [main] INFO io.druid.initialization.Initialization - Adding local module[class com.yahoo.mobile.mdi.druid.firehose.cms.CMSDruidModule] 1448 [main] INFO io.druid.initialization.Initialization - Adding local module[class com.yahoo.druid.sketch.module.SketchAggregatorModule] 1449 [main] INFO io.druid.initialization.Initialization - Adding local module[class com.yahoo.druid.sketch.module.SketchFileStorageFirehoseModule] 1956 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class com.metamx.emitter.core.LoggingEmitterConfig] from props[druid.emitter.logging.] as [LoggingEmitterConfig{loggerClass='com.metamx.emitter.core.LoggingEmitter', logLevel='info'}] 2051 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@1cdfe890] 2080 [main] INFO com.metamx.metrics.SysMonitor - Loading sigar native lib at tmpPath[/tmp/yay2750343450014951803yay/libsigar-amd64-linux.so] 2085 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[class com.metamx.metrics.SysMonitor, class com.metamx.metrics.JvmMonitor]}] 2109 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.DruidNode] from props[druid.] as [DruidNode{serviceName='overlord', host='be03.mdi.mep.ne1.yahoo.com:8085', port=8085}] 2120 [main] INFO io.druid.server.metrics.MetricsModule - Adding monitor[com.metamx.metrics.SysMonitor@19602f39] 2122 [main] INFO io.druid.server.metrics.MetricsModule - Adding monitor[com.metamx.metrics.JvmMonitor@4eee52c] 2184 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.TaskConfig] from props[druid.indexer.task.] as [io.druid.indexing.common.config.TaskConfig@3b9e714c] 2191 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.HttpClientModule$DruidHttpClientConfig] from props[druid.global.http.] as [io.druid.guice.HttpClientModule$DruidHttpClientConfig@50965003] 2358 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.indexing.IndexingServiceSelectorConfig] from props[druid.selectors.indexing.] as [io.druid.client.indexing.IndexingServiceSelectorConfig@1199831f] 2361 [main] INFO org.skife.config.ConfigurationObjectFactory - Assigning value [be04.mdi.mep.ne1.yahoo.com:2181,be05.mdi.mep.ne1.yahoo.com:2181,be06.mdi.mep.ne1.yahoo.com:2181] for [druid.zk.service.host] on [io.druid.curator.CuratorConfig#getZkHosts()] 2363 [main] INFO org.skife.config.ConfigurationObjectFactory - Assigning default value [30000] for [druid.zk.service.sessionTimeoutMs] on [io.druid.curator.CuratorConfig#getZkSessionTimeoutMs()] 2363 [main] INFO org.skife.config.ConfigurationObjectFactory - Assigning value [true] for [druid.curator.compress] on [io.druid.curator.CuratorConfig#enableCompression()] 2444 [main] WARN org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29 2486 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.CuratorDiscoveryConfig] from props[druid.discovery.curator.] as [io.druid.server.initialization.CuratorDiscoveryConfig@6f052145] 2644 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.RetryPolicyConfig] from props[druid.peon.taskActionClient.retry.] as [io.druid.indexing.common.RetryPolicyConfig@651293a0] 2647 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.storage.hdfs.HdfsDataSegmentPusherConfig] from props[druid.storage.] as [io.druid.storage.hdfs.HdfsDataSegmentPusherConfig@5fb2943c] 2664 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.DruidServerConfig] from props[druid.server.] as [io.druid.client.DruidServerConfig@5c1814e6] 2669 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.BatchDataSegmentAnnouncerConfig] from props[druid.announcer.] as [io.druid.server.initialization.BatchDataSegmentAnnouncerConfig@6398523f] 2670 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.loadQueuePath] on [io.druid.server.initialization.ZkPathsConfig#getLoadQueuePath()] 2670 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.indexer.announcementsPath] on [io.druid.server.initialization.ZkPathsConfig#getIndexerAnnouncementPath()] 2670 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.indexer.tasksPath] on [io.druid.server.initialization.ZkPathsConfig#getIndexerTaskPath()] 2671 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.coordinatorPath] on [io.druid.server.initialization.ZkPathsConfig#getCoordinatorPath()] 2671 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.connectorPath] on [io.druid.server.initialization.ZkPathsConfig#getConnectorPath()] 2671 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.propertiesPath] on [io.druid.server.initialization.ZkPathsConfig#getPropertiesPath()] 2671 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.indexer.statusPath] on [io.druid.server.initialization.ZkPathsConfig#getIndexerStatusPath()] 2671 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.indexer.leaderLatchPath] on [io.druid.server.initialization.ZkPathsConfig#getIndexerLeaderLatchPath()] 2671 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.liveSegmentsPath] on [io.druid.server.initialization.ZkPathsConfig#getLiveSegmentsPath()] 2672 [main] INFO org.skife.config.ConfigurationObjectFactory - Assigning value [/druid] for [druid.zk.paths.base] on [io.druid.server.initialization.ZkPathsConfig#getZkBasePath()] 2672 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.announcementsPath] on [io.druid.server.initialization.ZkPathsConfig#getAnnouncementsPath()] 2672 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.zk.paths.servedSegmentsPath] on [io.druid.server.initialization.ZkPathsConfig#getServedSegmentsPath()] 2702 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.coordination.DataSegmentAnnouncerProvider] from props[druid.announcer.] as [io.druid.server.coordination.BatchDataSegmentAnnouncerProvider@29b0baa8] 2708 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.client.ServerInventoryViewProvider] from props[druid.announcer.] as [io.druid.client.BatchServerInventoryViewProvider@f6fac55] 2717 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.QueryConfig] from props[druid.query.] as [io.druid.query.QueryConfig@1f50ad9a] 2724 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.search.search.SearchQueryConfig] from props[druid.query.search.] as [io.druid.query.search.search.SearchQueryConfig@6469048a] 2732 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.groupby.GroupByQueryConfig] from props[druid.query.groupBy.] as [io.druid.query.groupby.GroupByQueryConfig@48ad0abe] 2732 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.computation.buffer.size, ${base_path}.buffer.sizeBytes] on [io.druid.server.DruidProcessingConfig#intermediateComputeSizeBytes()] 2733 [main] INFO org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.numThreads] on [io.druid.server.DruidProcessingConfig#getNumThreads()] 2733 [main] INFO org.skife.config.ConfigurationObjectFactory - Assigning default value [processing-%s] for [${base_path}.formatString] on [com.metamx.common.concurrent.ExecutorServiceConfig#getFormatString()] 2751 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.topn.TopNQueryConfig] from props[druid.query.topN.] as [io.druid.query.topn.TopNQueryConfig@7b5ac5d0] 2760 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.server.log.RequestLoggerProvider] from props[druid.request.logging.] as [io.druid.server.log.NoopRequestLoggerProvider@77c4e567] 2767 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ServerConfig] from props[druid.server.http.] as [io.druid.server.initialization.ServerConfig@6ed0b159] 2773 [main] INFO org.eclipse.jetty.util.log - Logging initialized @2987ms 2853 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.emitter.core.LoggingEmitter.start()] on object[com.metamx.emitter.core.LoggingEmitter@73b0a5b]. 2853 [main] INFO com.metamx.emitter.core.LoggingEmitter - Start: started [true] 2853 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.emitter.service.ServiceEmitter.start()] on object[com.metamx.emitter.service.ServiceEmitter@58849313]. 2854 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.metrics.MonitorScheduler.start()] on object[com.metamx.metrics.MonitorScheduler@67d3f1cb]. 2859 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.http.client.HttpClient.start()] on object[com.metamx.http.client.HttpClient@5d2c8a22]. 2859 [main] INFO io.druid.curator.CuratorModule - Starting Curator 2859 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 2866 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT 2866 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=be03.mdi.mep.ne1.yahoo.com 2866 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.7.0_55 2866 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation 2866 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=/home/y/libexec64/jdk1.7.0/jre 2866 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/home/y/conf/mdi_druid/overlord/::/home/y/libexec/mdi_druid/lib/javax.servlet-api-3.1.0.jar:/home/y/libexec/mdi_druid/lib/jets3t-0.9.0.jar:/home/y/libexec/mdi_druid/lib/druid-common-0.6.105.jar:/home/y/libexec/mdi_druid/lib/hadoop-client-0.23.10.jar:/home/y/libexec/mdi_druid/lib/commons-cli-1.2.jar:/home/y/libexec/mdi_druid/lib/bytebuffer-collections-0.0.2.jar:/home/y/libexec/mdi_druid/lib/grizzly-rcm-2.1.1.jar:/home/y/libexec/mdi_druid/lib/jersey-json-1.17.jar:/home/y/libexec/mdi_druid/lib/jettison-1.1.jar:/home/y/libexec/mdi_druid/lib/geoip2-0.4.0.jar:/home/y/libexec/mdi_druid/lib/jetty-http-9.1.4.v20140401.jar:/home/y/libexec/mdi_druid/lib/java-xmlbuilder-0.4.jar:/home/y/libexec/mdi_druid/lib/json-simple-1.1.jar:/home/y/libexec/mdi_druid/lib/compress-lzf-0.8.4.jar:/home/y/libexec/mdi_druid/lib/geronimo-jms_1.1_spec-1.1.1.jar:/home/y/libexec/mdi_druid/lib/commons-el-1.0.jar:/home/y/libexec/mdi_druid/lib/cloud-messaging-common-0.3.jar:/home/y/libexec/mdi_druid/lib/jackson-core-2.3.1.jar:/home/y/libexec/mdi_druid/lib/jackson-databind-2.3.1.jar:/home/y/libexec/mdi_druid/lib/jackson-xc-1.9.2.jar:/home/y/libexec/mdi_druid/lib/aopalliance-1.0.jar:/home/y/libexec/mdi_druid/lib/jackson-jaxrs-json-provider-2.2.3.jar:/home/y/libexec/mdi_druid/lib/sherpa_client_plugin-0.6.0.0.jar:/home/y/libexec/mdi_druid/lib/aether-util-0.9.0.M2.jar:/home/y/libexec/mdi_druid/lib/guice-4.0-beta.jar:/home/y/libexec/mdi_druid/lib/guice-multibindings-4.0-beta.jar:/home/y/libexec/mdi_druid/lib/druid-processing-0.6.105.jar:/home/y/libexec/mdi_druid/lib/aws-java-sdk-1.6.0.1.jar:/home/y/libexec/mdi_druid/lib/aether-api-0.9.0.M2.jar:/home/y/libexec/mdi_druid/lib/hadoop-mapreduce-client-core-0.23.10.jar:/home/y/libexec/mdi_druid/lib/management-api-3.0.0-b012.jar:/home/y/libexec/mdi_druid/lib/jmxri-1.2.1.jar:/home/y/libexec/mdi_druid/lib/hadoop-yarn-common-0.23.10.jar:/home/y/libexec/mdi_druid/lib/classmate-0.8.0.jar:/home/y/libexec/mdi_druid/lib/irc-api-1.0-0011.jar:/home/y/libexec/mdi_druid/lib/xmlenc-0.52.jar:/home/y/libexec/mdi_druid/lib/jackson-jaxrs-1.9.2.jar:/home/y/libexec/mdi_druid/lib/jmxtools-1.2.1.jar:/home/y/libexec/mdi_druid/lib/slf4j-api-1.7.5.jar:/home/y/libexec/mdi_druid/lib/commons-dbcp-1.4.jar:/home/y/libexec/mdi_druid/lib/activation-1.1.jar:/home/y/libexec/mdi_druid/lib/curator-recipes-2.4.0.jar:/home/y/libexec/mdi_druid/lib/jackson-mapper-asl-1.9.2.jar:/home/y/libexec/mdi_druid/lib/junit-4.10.jar:/home/y/libexec/mdi_druid/lib/hadoop-mapreduce-client-common-0.23.10.jar:/home/y/libexec/mdi_druid/lib/jsr305-2.0.1.jar:/home/y/libexec/mdi_druid/lib/oro-2.0.8.jar:/home/y/libexec/mdi_druid/lib/hadoop-yarn-api-0.23.10.jar:/home/y/libexec/mdi_druid/lib/Common-1.0.0.0.jar:/home/y/libexec/mdi_druid/lib/commons-codec-1.7.jar:/home/y/libexec/mdi_druid/lib/javax.servlet-3.0.jar:/home/y/libexec/mdi_druid/lib/http-client-0.9.4.jar:/home/y/libexec/mdi_druid/lib/jersey-test-framework-core-1.8.jar:/home/y/libexec/mdi_druid/lib/tesla-aether-0.0.5.jar:/home/y/libexec/mdi_druid/lib/zookeeper-3.4.3.jar:/home/y/libexec/mdi_druid/lib/grizzly-http-2.1.1.jar:/home/y/libexec/mdi_druid/lib/jersey-servlet-1.17.1.jar:/home/y/libexec/mdi_druid/lib/jms-1.1.jar:/home/y/libexec/mdi_druid/lib/jul-to-slf4j-1.7.5.jar:/home/y/libexec/mdi_druid/lib/asm-3.1.jar:/home/y/libexec/mdi_druid/lib/druid-hdfs-storage-0.6.105.jar:/home/y/libexec/mdi_druid/lib/maxminddb-0.2.0.jar:/home/y/libexec/mdi_druid/lib/cloud-messaging-client-0.3.jar:/home/y/libexec/mdi_druid/lib/maven-settings-3.1.1.jar:/home/y/libexec/mdi_druid/lib/aether-connector-file-0.9.0.M2.jar:/home/y/libexec/mdi_druid/lib/javax.inject-1.jar:/home/y/libexec/mdi_druid/lib/org.abego.treelayout.core-1.0.1.jar:/home/y/libexec/mdi_druid/lib/jersey-client-1.17.jar:/home/y/libexec/mdi_druid/lib/hadoop-annotations-0.23.10.jar:/home/y/libexec/mdi_druid/lib/hadoop-hdfs-0.23.10.jar:/home/y/libexec/mdi_druid/lib/druid-services-0.6.105.jar:/home/y/libexec/mdi_druid/lib/netty-3.9.0.Final.jar:/home/y/libexec/mdi_druid/lib/plexus-interpolation-1.19.jar:/home/y/libexec/mdi_druid/lib/stax-api-1.0-2.jar:/home/y/libexec/mdi_druid/lib/druid-hll-0.6.105.jar:/home/y/libexec/mdi_druid/lib/aether-connector-okhttp-0.0.9.jar:/home/y/libexec/mdi_druid/lib/okhttp-1.0.2.jar:/home/y/libexec/mdi_druid/lib/jackson-datatype-joda-2.2.3.jar:/home/y/libexec/mdi_druid/lib/grizzly-http-server-2.1.1.jar:/home/y/libexec/mdi_druid/lib/druid-api-0.1.11.jar:/home/y/libexec/mdi_druid/lib/jersey-test-framework-grizzly2-1.8.jar:/home/y/libexec/mdi_druid/lib/emitter-0.2.11.jar:/home/y/libexec/mdi_druid/lib/yjava_yca-0.20.163.jar:/home/y/libexec/mdi_druid/lib/trove4j-3.0.3.jar:/home/y/libexec/mdi_druid/lib/jaxb-impl-2.2.3-1.jar:/home/y/libexec/mdi_druid/lib/jdbi-2.32.jar:/home/y/libexec/mdi_druid/lib/maven-repository-metadata-3.1.1.jar:/home/y/libexec/mdi_druid/lib/jetty-continuation-9.1.4.v20140401.jar:/home/y/libexec/mdi_druid/lib/jackson-datatype-guava-2.2.3.jar:/home/y/libexec/mdi_druid/lib/sigar-1.6.5.132.jar:/home/y/libexec/mdi_druid/lib/curator-client-2.4.0.jar:/home/y/libexec/mdi_druid/lib/Druid-1.0.0.0.jar:/home/y/libexec/mdi_druid/lib/grizzly-http-servlet-2.1.1.jar:/home/y/libexec/mdi_druid/lib/jetty-servlets-9.1.4.v20140401.jar:/home/y/libexec/mdi_druid/lib/commons-math-2.1.jar:/home/y/libexec/mdi_druid/lib/httpclient-4.3.2.jar:/home/y/libexec/mdi_druid/lib/server-metrics-0.0.9.jar:/home/y/libexec/mdi_druid/lib/extendedset-1.3.4.jar:/home/y/libexec/mdi_druid/lib/commons-logging-1.1.3.jar:/home/y/libexec/mdi_druid/lib/activemq-protobuf-1.1.jar:/home/y/libexec/mdi_druid/lib/netty-3.2.4.Final.jar:/home/y/libexec/mdi_druid/lib/jboss-logging-3.1.1.GA.jar:/home/y/libexec/mdi_druid/lib/commons-configuration-1.8.jar:/home/y/libexec/mdi_druid/lib/gson-1.7.2.jar:/home/y/libexec/mdi_druid/lib/google-http-client-jackson2-1.15.0-rc.jar:/home/y/libexec/mdi_druid/lib/guava-14.0.1.jar:/home/y/libexec/mdi_druid/lib/jaxb-api-2.2.2.jar:/home/y/libexec/mdi_druid/lib/grizzly-framework-2.1.1-tests.jar:/home/y/libexec/mdi_druid/lib/airline-0.6.jar:/home/y/libexec/mdi_druid/lib/aether-impl-0.9.0.M2.jar:/home/y/libexec/mdi_druid/lib/druid-server-0.6.105.jar:/home/y/libexec/mdi_druid/lib/hadoop-mapreduce-client-jobclient-0.23.10.jar:/home/y/libexec/mdi_druid/lib/hadoop-common-0.23.10.jar:/home/y/libexec/mdi_druid/lib/hibernate-validator-5.0.1.Final.jar:/home/y/libexec/mdi_druid/lib/stats_sketch-2.0.0.74121.jar:/home/y/libexec/mdi_druid/lib/hadoop-mapreduce-client-shuffle-0.23.10.jar:/home/y/libexec/mdi_druid/lib/hamcrest-core-1.1.jar:/home/y/libexec/mdi_druid/lib/log4j-1.2.15.jar:/home/y/libexec/mdi_druid/lib/jetty-servlet-9.1.4.v20140401.jar:/home/y/libexec/mdi_druid/lib/grizzly-framework-2.1.1.jar:/home/y/libexec/mdi_druid/lib/jetty-util-9.1.4.v20140401.jar:/home/y/libexec/mdi_druid/lib/yjava_daemon.jar:/home/y/libexec/mdi_druid/lib/plexus-utils-3.0.15.jar:/home/y/libexec/mdi_druid/lib/jackson-jaxrs-base-2.2.3.jar:/home/y/libexec/mdi_druid/lib/config-magic-0.9.jar:/home/y/libexec/mdi_druid/lib/commons-lang-2.3.jar:/home/y/libexec/mdi_druid/lib/druid-indexing-service-0.6.105.jar:/home/y/libexec/mdi_druid/lib/druid-indexing-hadoop-0.6.105.jar:/home/y/libexec/mdi_druid/lib/gmbal-api-only-3.0.0-b023.jar:/home/y/libexec/mdi_druid/lib/jersey-grizzly2-1.8.jar:/home/y/libexec/mdi_druid/lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar:/home/y/libexec/mdi_druid/lib/curator-x-discovery-2.4.0.jar:/home/y/libexec/mdi_druid/lib/jackson-core-asl-1.9.2.jar:/home/y/libexec/mdi_druid/lib/jdiff-1.0.9.jar:/home/y/libexec/mdi_druid/lib/hadoop-mapreduce-client-app-0.23.10.jar:/home/y/libexec/mdi_druid/lib/jackson-module-jaxb-annotations-2.2.3.jar:/home/y/libexec/mdi_druid/lib/maven-model-builder-3.1.1.jar:/home/y/libexec/mdi_druid/lib/jackson-dataformat-smile-2.2.3.jar:/home/y/libexec/mdi_druid/lib/httpcore-4.3.2.jar:/home/y/libexec/mdi_druid/lib/jersey-core-1.17.1.jar:/home/y/libexec/mdi_druid/lib/xpp3-1.1.4c.jar:/home/y/libexec/mdi_druid/lib/jetty-io-9.1.4.v20140401.jar:/home/y/libexec/mdi_druid/lib/slf4j-simple-1.6.4.jar:/home/y/libexec/mdi_druid/lib/curator-framework-2.4.0.jar:/home/y/libexec/mdi_druid/lib/mail-1.4.7.jar:/home/y/libexec/mdi_druid/lib/java-util-0.25.5.jar:/home/y/libexec/mdi_druid/lib/mysql-connector-java-5.1.18.jar:/home/y/libexec/mdi_druid/lib/jcontrib_sherpa_client-0.3.204.jar:/home/y/libexec/mdi_druid/lib/commons-collections-3.2.1.jar:/home/y/libexec/mdi_druid/lib/wagon-provider-api-2.4.jar:/home/y/libexec/mdi_druid/lib/rhino-1.7R4.jar:/home/y/libexec/mdi_druid/lib/hadoop-yarn-server-common-0.23.10.jar:/home/y/libexec/mdi_druid/lib/json-20090211.jar:/home/y/libexec/mdi_druid/lib/jasypt-1.9.0.jar:/home/y/libexec/mdi_druid/lib/google-http-client-1.15.0-rc.jar:/home/y/libexec/mdi_druid/lib/aether-spi-0.9.0.M2.jar:/home/y/libexec/mdi_druid/lib/jersey-guice-1.17.1.jar:/home/y/libexec/mdi_druid/lib/commons-net-3.1.jar:/home/y/libexec/mdi_druid/lib/jetty-server-9.1.4.v20140401.jar:/home/y/libexec/mdi_druid/lib/commons-pool-1.6.jar:/home/y/libexec/mdi_druid/lib/jline-0.9.94.jar:/home/y/libexec/mdi_druid/lib/guice-servlet-4.0-beta.jar:/home/y/libexec/mdi_druid/lib/jetty-security-9.1.4.v20140401.jar:/home/y/libexec/mdi_druid/lib/slf4j-log4j12-1.6.2.jar:/home/y/libexec/mdi_druid/lib/commons-io-2.4.jar:/home/y/libexec/mdi_druid/lib/icu4j-4.8.1.jar:/home/y/libexec/mdi_druid/lib/maven-aether-provider-3.1.1.jar:/home/y/libexec/mdi_druid/lib/jersey-server-1.17.1.jar:/home/y/libexec/mdi_druid/lib/hadoop-auth-0.23.10.jar:/home/y/libexec/mdi_druid/lib/jackson-annotations-2.3.1.jar:/home/y/libexec/mdi_druid/lib/maven-settings-builder-3.1.1.jar:/home/y/libexec/mdi_druid/lib/opencsv-2.3.jar:/home/y/libexec/mdi_druid/lib/lz4-1.1.2.jar:/home/y/libexec/mdi_druid/lib/protobuf-java-2.4.1.jar:/home/y/libexec/mdi_druid/lib/avro-1.0.0.jar:/home/y/libexec/mdi_druid/lib/antlr4-runtime-4.0.jar:/home/y/libexec/mdi_druid/lib/maven-model-3.1.1.jar:/home/y/libexec/mdi_druid/lib/activemq-core-5.7.0.17.jar:/home/y/libexec/mdi_druid/lib/validation-api-1.1.0.Final.jar:/home/y/libexec/mdi_druid/lib/joda-time-2.1.jar:/home/y/libexec/mdi_druid/lib/spymemcached-2.8.4.jar 2866 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/home/y/lib64 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA> 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=2.6.32-431.3.1.el6.YAHOO.20140110.x86_64 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=nobody 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=/ 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/y/libexec/mdi_druid 2867 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=be04.mdi.mep.ne1.yahoo.com:2181,be05.mdi.mep.ne1.yahoo.com:2181,be06.mdi.mep.ne1.yahoo.com:2181 sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@73866914 2883 [main-SendThread()] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server /98.138.250.114:2181 2884 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.discovery.ServerDiscoverySelector.start() throws java.lang.Exception] on object[io.druid.curator.discovery.ServerDiscoverySelector@5b93ceb6]. 2887 [main-SendThread(be06.mdi.mep.ne1.yahoo.com:2181)] INFO org.apache.zookeeper.client.ZooKeeperSaslClient - Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration. 2894 [main-SendThread(be06.mdi.mep.ne1.yahoo.com:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to be06.mdi.mep.ne1.yahoo.com/98.138.250.114:2181, initiating session 2921 [main-SendThread(be06.mdi.mep.ne1.yahoo.com:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server be06.mdi.mep.ne1.yahoo.com/98.138.250.114:2181, sessionid = 0x345d566ea5f0020, negotiated timeout = 30000 2926 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 4044 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.announcement.Announcer.start()] on object[io.druid.curator.announcement.Announcer@55fa81]. 4046 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.client.ServerInventoryView.start() throws java.lang.Exception] on object[io.druid.client.BatchServerInventoryView@6504e8ae]. 4051 [main] INFO org.eclipse.jetty.server.Server - jetty-9.1.4.v20140401 4114 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/druid05.mdi.mep.ne1.yahoo.com:8081 4114 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for druid05.mdi.mep.ne1.yahoo.com:8081, inventoryPath /druid/segments/druid05.mdi.mep.ne1.yahoo.com:8081 4115 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='druid05.mdi.mep.ne1.yahoo.com:8081', host='druid05.mdi.mep.ne1.yahoo.com:8081', maxSize=180000000000, tier='_default_tier', type='historical', priority='0'}] 4115 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/druid04.mdi.mep.ne1.yahoo.com:8081 4115 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for druid04.mdi.mep.ne1.yahoo.com:8081, inventoryPath /druid/segments/druid04.mdi.mep.ne1.yahoo.com:8081 4115 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='druid04.mdi.mep.ne1.yahoo.com:8081', host='druid04.mdi.mep.ne1.yahoo.com:8081', maxSize=180000000000, tier='_default_tier', type='historical', priority='0'}] 4116 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/be02.mdi.mep.ne1.yahoo.com:8083 4116 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for be02.mdi.mep.ne1.yahoo.com:8083, inventoryPath /druid/segments/be02.mdi.mep.ne1.yahoo.com:8083 4116 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='be02.mdi.mep.ne1.yahoo.com:8083', host='be02.mdi.mep.ne1.yahoo.com:8083', maxSize=300000000000, tier='_default_tier', type='realtime', priority='0'}] 4116 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/druid08.mdi.mep.ne1.yahoo.com:8081 4116 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for druid08.mdi.mep.ne1.yahoo.com:8081, inventoryPath /druid/segments/druid08.mdi.mep.ne1.yahoo.com:8081 4117 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='druid08.mdi.mep.ne1.yahoo.com:8081', host='druid08.mdi.mep.ne1.yahoo.com:8081', maxSize=180000000000, tier='_default_tier', type='historical', priority='0'}] 4117 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/druid07.mdi.mep.ne1.yahoo.com:8081 4117 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for druid07.mdi.mep.ne1.yahoo.com:8081, inventoryPath /druid/segments/druid07.mdi.mep.ne1.yahoo.com:8081 4117 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='druid07.mdi.mep.ne1.yahoo.com:8081', host='druid07.mdi.mep.ne1.yahoo.com:8081', maxSize=180000000000, tier='_default_tier', type='historical', priority='0'}] 4118 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/druid10.mdi.mep.ne1.yahoo.com:8081 4118 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for druid10.mdi.mep.ne1.yahoo.com:8081, inventoryPath /druid/segments/druid10.mdi.mep.ne1.yahoo.com:8081 4118 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='druid10.mdi.mep.ne1.yahoo.com:8081', host='druid10.mdi.mep.ne1.yahoo.com:8081', maxSize=180000000000, tier='_default_tier', type='historical', priority='0'}] 4118 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/druid09.mdi.mep.ne1.yahoo.com:8081 4118 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for druid09.mdi.mep.ne1.yahoo.com:8081, inventoryPath /druid/segments/druid09.mdi.mep.ne1.yahoo.com:8081 4118 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='druid09.mdi.mep.ne1.yahoo.com:8081', host='druid09.mdi.mep.ne1.yahoo.com:8081', maxSize=180000000000, tier='_default_tier', type='historical', priority='0'}] 4119 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/druid06.mdi.mep.ne1.yahoo.com:8081 4119 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for druid06.mdi.mep.ne1.yahoo.com:8081, inventoryPath /druid/segments/druid06.mdi.mep.ne1.yahoo.com:8081 4119 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='druid06.mdi.mep.ne1.yahoo.com:8081', host='druid06.mdi.mep.ne1.yahoo.com:8081', maxSize=180000000000, tier='_default_tier', type='historical', priority='0'}] 4119 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/druid03.mdi.mep.ne1.yahoo.com:8081 4120 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for druid03.mdi.mep.ne1.yahoo.com:8081, inventoryPath /druid/segments/druid03.mdi.mep.ne1.yahoo.com:8081 4120 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='druid03.mdi.mep.ne1.yahoo.com:8081', host='druid03.mdi.mep.ne1.yahoo.com:8081', maxSize=180000000000, tier='_default_tier', type='historical', priority='0'}] 4175 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T06:00:00.000Z_2014-05-10T07:00:00.000Z_2014-05-10T06:00:00.000Z] 4176 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T12:00:00.000Z_2014-05-10T13:00:00.000Z_2014-05-10T12:00:00.000Z] 4176 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T18:00:00.000Z_2014-05-08T19:00:00.000Z_2014-05-08T18:00:00.000Z] 4176 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-13T00:00:00.000Z_2014-05-13T01:00:00.000Z_2014-05-13T00:00:00.000Z] 4176 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T06:00:00.000Z_2014-05-09T07:00:00.000Z_2014-05-09T06:00:00.000Z] 4177 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T22:00:00.000Z_2014-05-08T23:00:00.000Z_2014-05-08T22:00:00.000Z] 4177 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T20:00:00.000Z_2014-05-11T21:00:00.000Z_2014-05-11T20:00:00.000Z] 4177 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T14:00:00.000Z_2014-05-09T15:00:00.000Z_2014-05-09T14:00:00.000Z] 4177 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T14:00:00.000Z_2014-05-11T15:00:00.000Z_2014-05-11T14:00:00.000Z] 4177 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T14:00:00.000Z_2014-05-10T15:00:00.000Z_2014-05-10T14:00:00.000Z] 4177 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T14:00:00.000Z_2014-05-12T15:00:00.000Z_2014-05-12T14:00:00.000Z] 4178 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T06:00:00.000Z_2014-05-12T07:00:00.000Z_2014-05-12T06:00:00.000Z] 4178 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T08:00:00.000Z_2014-05-08T09:00:00.000Z_2014-05-08T08:00:00.000Z] 4178 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T22:00:00.000Z_2014-05-12T23:00:00.000Z_2014-05-12T22:00:00.000Z] 4178 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid05.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T02:00:00.000Z_2014-05-09T03:00:00.000Z_2014-05-09T02:00:00.000Z] 4189 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T08:00:00.000Z_2014-05-12T09:00:00.000Z_2014-05-12T08:00:00.000Z] 4189 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T16:00:00.000Z_2014-05-10T17:00:00.000Z_2014-05-10T16:00:00.000Z] 4190 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T10:00:00.000Z_2014-05-08T11:00:00.000Z_2014-05-08T10:00:00.000Z] 4190 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T12:00:00.000Z_2014-05-09T13:00:00.000Z_2014-05-09T12:00:00.000Z] 4190 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T02:00:00.000Z_2014-05-11T03:00:00.000Z_2014-05-11T02:00:00.000Z] 4190 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T00:00:00.000Z_2014-05-08T01:00:00.000Z_2014-05-08T00:00:00.000Z] 4190 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T20:00:00.000Z_2014-05-09T21:00:00.000Z_2014-05-09T20:00:00.000Z] 4190 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T10:00:00.000Z_2014-05-11T11:00:00.000Z_2014-05-11T10:00:00.000Z] 4190 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T04:00:00.000Z_2014-05-10T05:00:00.000Z_2014-05-10T04:00:00.000Z] 4190 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T08:00:00.000Z_2014-05-09T09:00:00.000Z_2014-05-09T08:00:00.000Z] 4191 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T18:00:00.000Z_2014-05-09T19:00:00.000Z_2014-05-09T18:00:00.000Z] 4191 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T04:00:00.000Z_2014-05-09T05:00:00.000Z_2014-05-09T04:00:00.000Z] 4191 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T22:00:00.000Z_2014-05-11T23:00:00.000Z_2014-05-11T22:00:00.000Z] 4191 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T22:00:00.000Z_2014-05-08T23:00:00.000Z_2014-05-08T22:00:00.000Z] 4191 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T20:00:00.000Z_2014-05-12T21:00:00.000Z_2014-05-12T20:00:00.000Z] 4191 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid04.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T06:00:00.000Z_2014-05-08T07:00:00.000Z_2014-05-08T06:00:00.000Z] May 13, 2014 1:17:06 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider as a provider class May 13, 2014 1:17:06 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering io.druid.server.StatusResource as a root resource class May 13, 2014 1:17:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.17.1 02/28/2013 12:47 PM' 4199 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T23:00:00.000Z_2014-05-12T00:00:00.000Z_2014-05-11T23:00:00.000Z] 4199 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T09:00:00.000Z_2014-05-12T10:00:00.000Z_2014-05-12T09:00:00.000Z] 4199 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T11:00:00.000Z_2014-05-12T12:00:00.000Z_2014-05-12T11:00:00.000Z] 4199 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T03:00:00.000Z_2014-05-12T04:00:00.000Z_2014-05-12T03:00:00.000Z] 4199 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T05:00:00.000Z_2014-05-12T06:00:00.000Z_2014-05-12T05:00:00.000Z] 4199 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-13T01:00:00.000Z_2014-05-13T02:00:00.000Z_2014-05-13T01:00:00.000Z] 4200 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T23:00:00.000Z_2014-05-13T00:00:00.000Z_2014-05-12T23:00:00.000Z] 4200 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T13:00:00.000Z_2014-05-12T14:00:00.000Z_2014-05-12T13:00:00.000Z] 4200 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T01:00:00.000Z_2014-05-12T02:00:00.000Z_2014-05-12T01:00:00.000Z] 4200 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T15:00:00.000Z_2014-05-12T16:00:00.000Z_2014-05-12T15:00:00.000Z] 4200 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T19:00:00.000Z_2014-05-12T20:00:00.000Z_2014-05-12T19:00:00.000Z] 4200 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T07:00:00.000Z_2014-05-12T08:00:00.000Z_2014-05-12T07:00:00.000Z] 4200 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T17:00:00.000Z_2014-05-12T18:00:00.000Z_2014-05-12T17:00:00.000Z] 4200 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-12T21:00:00.000Z_2014-05-12T22:00:00.000Z_2014-05-12T21:00:00.000Z] 4226 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T05:00:00.000Z_2014-05-11T06:00:00.000Z_2014-05-11T05:00:00.000Z] 4226 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T19:00:00.000Z_2014-05-10T20:00:00.000Z_2014-05-10T19:00:00.000Z] 4226 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T01:00:00.000Z_2014-05-11T02:00:00.000Z_2014-05-11T01:00:00.000Z] 4226 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T17:00:00.000Z_2014-05-10T18:00:00.000Z_2014-05-10T17:00:00.000Z] 4226 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T09:00:00.000Z_2014-05-10T10:00:00.000Z_2014-05-10T09:00:00.000Z] 4226 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T23:00:00.000Z_2014-05-09T00:00:00.000Z_2014-05-08T23:00:00.000Z] 4226 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T17:00:00.000Z_2014-05-11T18:00:00.000Z_2014-05-11T17:00:00.000Z] 4226 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T15:00:00.000Z_2014-05-09T16:00:00.000Z_2014-05-09T15:00:00.000Z] 4227 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T19:00:00.000Z_2014-05-11T20:00:00.000Z_2014-05-11T19:00:00.000Z] 4227 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T13:00:00.000Z_2014-05-09T14:00:00.000Z_2014-05-09T13:00:00.000Z] 4227 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T15:00:00.000Z_2014-05-11T16:00:00.000Z_2014-05-11T15:00:00.000Z] 4227 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T09:00:00.000Z_2014-05-09T10:00:00.000Z_2014-05-09T09:00:00.000Z] 4227 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T03:00:00.000Z_2014-05-11T04:00:00.000Z_2014-05-11T03:00:00.000Z] 4227 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T13:00:00.000Z_2014-05-08T14:00:00.000Z_2014-05-08T13:00:00.000Z] 4227 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T15:00:00.000Z_2014-05-08T16:00:00.000Z_2014-05-08T15:00:00.000Z] 4227 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T21:00:00.000Z_2014-05-11T22:00:00.000Z_2014-05-11T21:00:00.000Z] 4228 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-07T21:00:00.000Z_2014-05-07T22:00:00.000Z_2014-05-07T21:00:00.000Z] 4228 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T05:00:00.000Z_2014-05-10T06:00:00.000Z_2014-05-10T05:00:00.000Z] 4228 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T13:00:00.000Z_2014-05-11T14:00:00.000Z_2014-05-11T13:00:00.000Z] 4228 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T07:00:00.000Z_2014-05-11T08:00:00.000Z_2014-05-11T07:00:00.000Z] 4228 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T07:00:00.000Z_2014-05-09T08:00:00.000Z_2014-05-09T07:00:00.000Z] 4228 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T17:00:00.000Z_2014-05-09T18:00:00.000Z_2014-05-09T17:00:00.000Z] 4228 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T05:00:00.000Z_2014-05-09T06:00:00.000Z_2014-05-09T05:00:00.000Z] 4228 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T23:00:00.000Z_2014-05-10T00:00:00.000Z_2014-05-09T23:00:00.000Z] 4229 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T13:00:00.000Z_2014-05-10T14:00:00.000Z_2014-05-10T13:00:00.000Z] 4229 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T15:00:00.000Z_2014-05-10T16:00:00.000Z_2014-05-10T15:00:00.000Z] 4229 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T01:00:00.000Z_2014-05-08T02:00:00.000Z_2014-05-08T01:00:00.000Z] 4229 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T01:00:00.000Z_2014-05-09T02:00:00.000Z_2014-05-09T01:00:00.000Z] 4229 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-07T23:00:00.000Z_2014-05-08T00:00:00.000Z_2014-05-07T23:00:00.000Z] 4229 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T11:00:00.000Z_2014-05-08T12:00:00.000Z_2014-05-08T11:00:00.000Z] 4229 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T09:00:00.000Z_2014-05-08T10:00:00.000Z_2014-05-08T09:00:00.000Z] 4229 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T19:00:00.000Z_2014-05-08T20:00:00.000Z_2014-05-08T19:00:00.000Z] 4230 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T11:00:00.000Z_2014-05-09T12:00:00.000Z_2014-05-09T11:00:00.000Z] 4230 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T03:00:00.000Z_2014-05-08T04:00:00.000Z_2014-05-08T03:00:00.000Z] 4230 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T07:00:00.000Z_2014-05-10T08:00:00.000Z_2014-05-10T07:00:00.000Z] 4230 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T11:00:00.000Z_2014-05-10T12:00:00.000Z_2014-05-10T11:00:00.000Z] 4230 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T19:00:00.000Z_2014-05-09T20:00:00.000Z_2014-05-09T19:00:00.000Z] 4230 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T21:00:00.000Z_2014-05-09T22:00:00.000Z_2014-05-09T21:00:00.000Z] 4230 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T21:00:00.000Z_2014-05-08T22:00:00.000Z_2014-05-08T21:00:00.000Z] 4230 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T11:00:00.000Z_2014-05-11T12:00:00.000Z_2014-05-11T11:00:00.000Z] 4231 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-09T03:00:00.000Z_2014-05-09T04:00:00.000Z_2014-05-09T03:00:00.000Z] 4231 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T17:00:00.000Z_2014-05-08T18:00:00.000Z_2014-05-08T17:00:00.000Z] 4231 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T03:00:00.000Z_2014-05-10T04:00:00.000Z_2014-05-10T03:00:00.000Z] 4231 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T05:00:00.000Z_2014-05-08T06:00:00.000Z_2014-05-08T05:00:00.000Z] 4231 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T21:00:00.000Z_2014-05-10T22:00:00.000Z_2014-05-10T21:00:00.000Z] 4231 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-11T09:00:00.000Z_2014-05-11T10:00:00.000Z_2014-05-11T09:00:00.000Z] 4231 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T01:00:00.000Z_2014-05-10T02:00:00.000Z_2014-05-10T01:00:00.000Z] 4231 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-08T07:00:00.000Z_2014-05-08T08:00:00.000Z_2014-05-08T07:00:00.000Z] 4232 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[be02.mdi.mep.ne1.yahoo.com:8083] added segment[wiki_irc_2014-05-10T23:00:00.000Z_2014-05-11T00:00:00.000Z_2014-05-10T23:00:00.000Z] 4241 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T00:00:00.000Z_2014-05-11T01:00:00.000Z_2014-05-11T00:00:00.000Z] 4241 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T02:00:00.000Z_2014-05-12T03:00:00.000Z_2014-05-12T02:00:00.000Z] 4241 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T02:00:00.000Z_2014-05-10T03:00:00.000Z_2014-05-10T02:00:00.000Z] 4241 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T10:00:00.000Z_2014-05-12T11:00:00.000Z_2014-05-12T10:00:00.000Z] 4241 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T14:00:00.000Z_2014-05-10T15:00:00.000Z_2014-05-10T14:00:00.000Z] 4242 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T00:00:00.000Z_2014-05-08T01:00:00.000Z_2014-05-08T00:00:00.000Z] 4242 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T10:00:00.000Z_2014-05-11T11:00:00.000Z_2014-05-11T10:00:00.000Z] 4242 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T22:00:00.000Z_2014-05-10T23:00:00.000Z_2014-05-10T22:00:00.000Z] 4242 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T06:00:00.000Z_2014-05-10T07:00:00.000Z_2014-05-10T06:00:00.000Z] 4242 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T12:00:00.000Z_2014-05-08T13:00:00.000Z_2014-05-08T12:00:00.000Z] 4242 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T18:00:00.000Z_2014-05-10T19:00:00.000Z_2014-05-10T18:00:00.000Z] 4242 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T16:00:00.000Z_2014-05-12T17:00:00.000Z_2014-05-12T16:00:00.000Z] 4243 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T22:00:00.000Z_2014-05-09T23:00:00.000Z_2014-05-09T22:00:00.000Z] 4243 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T16:00:00.000Z_2014-05-11T17:00:00.000Z_2014-05-11T16:00:00.000Z] 4243 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid08.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T20:00:00.000Z_2014-05-12T21:00:00.000Z_2014-05-12T20:00:00.000Z] 4252 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T12:00:00.000Z_2014-05-12T13:00:00.000Z_2014-05-12T12:00:00.000Z] 4253 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T20:00:00.000Z_2014-05-10T21:00:00.000Z_2014-05-10T20:00:00.000Z] 4253 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T04:00:00.000Z_2014-05-08T05:00:00.000Z_2014-05-08T04:00:00.000Z] 4253 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T16:00:00.000Z_2014-05-10T17:00:00.000Z_2014-05-10T16:00:00.000Z] 4253 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T10:00:00.000Z_2014-05-09T11:00:00.000Z_2014-05-09T10:00:00.000Z] 4253 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T06:00:00.000Z_2014-05-12T07:00:00.000Z_2014-05-12T06:00:00.000Z] 4253 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T16:00:00.000Z_2014-05-09T17:00:00.000Z_2014-05-09T16:00:00.000Z] 4253 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T04:00:00.000Z_2014-05-10T05:00:00.000Z_2014-05-10T04:00:00.000Z] 4253 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T22:00:00.000Z_2014-05-10T23:00:00.000Z_2014-05-10T22:00:00.000Z] 4254 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T20:00:00.000Z_2014-05-08T21:00:00.000Z_2014-05-08T20:00:00.000Z] 4254 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T00:00:00.000Z_2014-05-09T01:00:00.000Z_2014-05-09T00:00:00.000Z] 4254 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T04:00:00.000Z_2014-05-09T05:00:00.000Z_2014-05-09T04:00:00.000Z] 4254 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T10:00:00.000Z_2014-05-10T11:00:00.000Z_2014-05-10T10:00:00.000Z] 4254 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T14:00:00.000Z_2014-05-08T15:00:00.000Z_2014-05-08T14:00:00.000Z] 4254 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T02:00:00.000Z_2014-05-08T03:00:00.000Z_2014-05-08T02:00:00.000Z] 4254 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid07.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T16:00:00.000Z_2014-05-12T17:00:00.000Z_2014-05-12T16:00:00.000Z] 4264 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T20:00:00.000Z_2014-05-10T21:00:00.000Z_2014-05-10T20:00:00.000Z] 4264 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T02:00:00.000Z_2014-05-10T03:00:00.000Z_2014-05-10T02:00:00.000Z] 4264 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T04:00:00.000Z_2014-05-08T05:00:00.000Z_2014-05-08T04:00:00.000Z] 4264 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T08:00:00.000Z_2014-05-12T09:00:00.000Z_2014-05-12T08:00:00.000Z] 4264 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T04:00:00.000Z_2014-05-12T05:00:00.000Z_2014-05-12T04:00:00.000Z] 4264 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T20:00:00.000Z_2014-05-11T21:00:00.000Z_2014-05-11T20:00:00.000Z] 4264 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T12:00:00.000Z_2014-05-09T13:00:00.000Z_2014-05-09T12:00:00.000Z] 4264 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T10:00:00.000Z_2014-05-08T11:00:00.000Z_2014-05-08T10:00:00.000Z] 4265 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T14:00:00.000Z_2014-05-12T15:00:00.000Z_2014-05-12T14:00:00.000Z] 4265 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T16:00:00.000Z_2014-05-08T17:00:00.000Z_2014-05-08T16:00:00.000Z] 4265 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T20:00:00.000Z_2014-05-09T21:00:00.000Z_2014-05-09T20:00:00.000Z] 4265 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T12:00:00.000Z_2014-05-11T13:00:00.000Z_2014-05-11T12:00:00.000Z] 4265 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T00:00:00.000Z_2014-05-09T01:00:00.000Z_2014-05-09T00:00:00.000Z] 4265 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T02:00:00.000Z_2014-05-08T03:00:00.000Z_2014-05-08T02:00:00.000Z] 4265 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T22:00:00.000Z_2014-05-12T23:00:00.000Z_2014-05-12T22:00:00.000Z] 4265 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid10.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T02:00:00.000Z_2014-05-09T03:00:00.000Z_2014-05-09T02:00:00.000Z] 4274 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T18:00:00.000Z_2014-05-11T19:00:00.000Z_2014-05-11T18:00:00.000Z] 4274 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T18:00:00.000Z_2014-05-08T19:00:00.000Z_2014-05-08T18:00:00.000Z] 4274 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T02:00:00.000Z_2014-05-11T03:00:00.000Z_2014-05-11T02:00:00.000Z] 4274 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T10:00:00.000Z_2014-05-09T11:00:00.000Z_2014-05-09T10:00:00.000Z] 4274 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T00:00:00.000Z_2014-05-10T01:00:00.000Z_2014-05-10T00:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T18:00:00.000Z_2014-05-12T19:00:00.000Z_2014-05-12T18:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T06:00:00.000Z_2014-05-09T07:00:00.000Z_2014-05-09T06:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T22:00:00.000Z_2014-05-11T23:00:00.000Z_2014-05-11T22:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T10:00:00.000Z_2014-05-10T11:00:00.000Z_2014-05-10T10:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T04:00:00.000Z_2014-05-11T05:00:00.000Z_2014-05-11T04:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T18:00:00.000Z_2014-05-10T19:00:00.000Z_2014-05-10T18:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T08:00:00.000Z_2014-05-11T09:00:00.000Z_2014-05-11T08:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T22:00:00.000Z_2014-05-09T23:00:00.000Z_2014-05-09T22:00:00.000Z] 4275 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T16:00:00.000Z_2014-05-11T17:00:00.000Z_2014-05-11T16:00:00.000Z] 4276 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid09.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T08:00:00.000Z_2014-05-08T09:00:00.000Z_2014-05-08T08:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T12:00:00.000Z_2014-05-12T13:00:00.000Z_2014-05-12T12:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T18:00:00.000Z_2014-05-11T19:00:00.000Z_2014-05-11T18:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-07T22:00:00.000Z_2014-05-07T23:00:00.000Z_2014-05-07T22:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-13T00:00:00.000Z_2014-05-13T01:00:00.000Z_2014-05-13T00:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T04:00:00.000Z_2014-05-12T05:00:00.000Z_2014-05-12T04:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T14:00:00.000Z_2014-05-11T15:00:00.000Z_2014-05-11T14:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T16:00:00.000Z_2014-05-09T17:00:00.000Z_2014-05-09T16:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T16:00:00.000Z_2014-05-08T17:00:00.000Z_2014-05-08T16:00:00.000Z] 4285 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T00:00:00.000Z_2014-05-10T01:00:00.000Z_2014-05-10T00:00:00.000Z] 4286 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T20:00:00.000Z_2014-05-08T21:00:00.000Z_2014-05-08T20:00:00.000Z] 4286 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T08:00:00.000Z_2014-05-10T09:00:00.000Z_2014-05-10T08:00:00.000Z] 4286 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T00:00:00.000Z_2014-05-12T01:00:00.000Z_2014-05-12T00:00:00.000Z] 4286 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T04:00:00.000Z_2014-05-11T05:00:00.000Z_2014-05-11T04:00:00.000Z] 4286 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T08:00:00.000Z_2014-05-11T09:00:00.000Z_2014-05-11T08:00:00.000Z] 4286 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T06:00:00.000Z_2014-05-11T07:00:00.000Z_2014-05-11T06:00:00.000Z] 4286 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid06.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T06:00:00.000Z_2014-05-08T07:00:00.000Z_2014-05-08T06:00:00.000Z] 4295 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T00:00:00.000Z_2014-05-11T01:00:00.000Z_2014-05-11T00:00:00.000Z] 4295 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T02:00:00.000Z_2014-05-12T03:00:00.000Z_2014-05-12T02:00:00.000Z] 4295 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T12:00:00.000Z_2014-05-10T13:00:00.000Z_2014-05-10T12:00:00.000Z] 4295 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-07T22:00:00.000Z_2014-05-07T23:00:00.000Z_2014-05-07T22:00:00.000Z] 4296 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T10:00:00.000Z_2014-05-12T11:00:00.000Z_2014-05-12T10:00:00.000Z] 4296 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T14:00:00.000Z_2014-05-09T15:00:00.000Z_2014-05-09T14:00:00.000Z] 4296 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T12:00:00.000Z_2014-05-11T13:00:00.000Z_2014-05-11T12:00:00.000Z] 4296 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T18:00:00.000Z_2014-05-12T19:00:00.000Z_2014-05-12T18:00:00.000Z] 4296 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-10T08:00:00.000Z_2014-05-10T09:00:00.000Z_2014-05-10T08:00:00.000Z] 4296 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T08:00:00.000Z_2014-05-09T09:00:00.000Z_2014-05-09T08:00:00.000Z] 4296 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-09T18:00:00.000Z_2014-05-09T19:00:00.000Z_2014-05-09T18:00:00.000Z] 4296 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-12T00:00:00.000Z_2014-05-12T01:00:00.000Z_2014-05-12T00:00:00.000Z] 4297 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T14:00:00.000Z_2014-05-08T15:00:00.000Z_2014-05-08T14:00:00.000Z] 4297 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-08T12:00:00.000Z_2014-05-08T13:00:00.000Z_2014-05-08T12:00:00.000Z] 4297 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server[druid03.mdi.mep.ne1.yahoo.com:8081] added segment[wiki_irc_2014-05-11T06:00:00.000Z_2014-05-11T07:00:00.000Z_2014-05-11T06:00:00.000Z] May 13, 2014 1:17:06 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" May 13, 2014 1:17:07 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding io.druid.server.QueryResource to GuiceInstantiatedComponentProvider May 13, 2014 1:17:07 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding io.druid.indexing.worker.executor.ChatHandlerResource to GuiceInstantiatedComponentProvider May 13, 2014 1:17:07 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding io.druid.server.StatusResource to GuiceManagedComponentProvider with the scope "Undefined" 4976 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@8ddadc8{/,null,AVAILABLE} 4985 [main] INFO org.eclipse.jetty.server.ServerConnector - Started ServerConnector@3562626{HTTP/1.1}{0.0.0.0:8085} 4986 [main] INFO org.eclipse.jetty.server.Server - Started @5200ms 4986 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.server.coordination.AbstractDataSegmentAnnouncer.start()] on object[io.druid.server.coordination.BatchDataSegmentAnnouncer@5692fc6]. 4986 [main] INFO io.druid.server.coordination.AbstractDataSegmentAnnouncer - Announcing self[DruidServerMetadata{name='be03.mdi.mep.ne1.yahoo.com:8085', host='be03.mdi.mep.ne1.yahoo.com:8085', maxSize=0, tier='_default_tier', type='indexer-executor', priority='0'}] at [/druid/announcements/be03.mdi.mep.ne1.yahoo.com:8085] 5028 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.indexing.worker.executor.ExecutorLifecycle.start()] on object[io.druid.indexing.worker.executor.ExecutorLifecycle@44f0adaf]. 5047 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/be03.mdi.mep.ne1.yahoo.com:8085 5047 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Starting inventory cache for be03.mdi.mep.ne1.yahoo.com:8085, inventoryPath /druid/segments/be03.mdi.mep.ne1.yahoo.com:8085 5047 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='be03.mdi.mep.ne1.yahoo.com:8085', host='be03.mdi.mep.ne1.yahoo.com:8085', maxSize=0, tier='_default_tier', type='indexer-executor', priority='0'}] 5156 [main] INFO io.druid.server.initialization.PropertiesModule - Loading properties from runtime.properties 5180 [main] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[], localRepository='/home/y/tmp/.m2/repository', remoteRepositories=[]}] 5216 [main] INFO io.druid.indexing.worker.executor.ExecutorLifecycle - Running with task: { "type" : "index_hadoop", "id" : "index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z", "config" : { "dataSource" : "flickr1_sketch", "timestampSpec" : { "column" : "timestamp", "format" : "YYYYMMdd" }, "dataSpec" : { "format" : "tsv", "delimiter" : "\t", "columns" : [ "timestamp", "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type", "bcookieSketch" ], "dimensions" : [ "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type" ], "spatialDimensions" : [ ] }, "granularitySpec" : { "type" : "uniform", "gran" : "DAY", "intervals" : [ "2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z" ] }, "pathSpec" : { "type" : "static", "paths" : "hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv" }, "workingPath" : null, "segmentOutputPath" : null, "version" : "2014-05-13T01:17:01.684Z", "partitionsSpec" : null, "leaveIntermediate" : false, "cleanupOnFailure" : true, "shardSpecs" : { }, "overwriteFiles" : false, "rollupSpec" : { "aggs" : [ { "type" : "count", "name" : "count" }, { "type" : "sketchMerge", "name" : "bcookieSketch", "fieldName" : "bcookieSketch", "size" : 65536, "filter" : { "type" : "true" } } ], "rollupGranularity" : { "type" : "none" }, "rowFlushBoundary" : 500000 }, "updaterJobSpec" : null, "ignoreInvalidRows" : false }, "hadoopDependencyCoordinates" : [ "org.apache.hadoop:hadoop-client:0.23.10" ], "groupId" : "index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z", "resource" : { "availabilityGroup" : "index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z", "requiredCapacity" : 1 }, "dataSource" : "flickr1_sketch" } 5220 [main] INFO io.druid.indexing.common.actions.RemoteTaskActionClient - Performing action for task[index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z]: LockTryAcquireAction{interval=2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z} 5230 [main] INFO io.druid.indexing.common.actions.RemoteTaskActionClient - Submitting action for task[index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z] to overlord[http://be03.mdi.mep.ne1.yahoo.com:4080/druid/indexer/v1/action]: LockTryAcquireAction{interval=2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z} 2014-05-13 01:17:07,330 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://be03.mdi.mep.ne1.yahoo.com:4080 2014-05-13 01:17:07,368 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://be03.mdi.mep.ne1.yahoo.com:4080 2014-05-13 01:17:07,369 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://be03.mdi.mep.ne1.yahoo.com:4080 2014-05-13 01:17:07,369 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://be03.mdi.mep.ne1.yahoo.com:4080 2014-05-13 01:17:07,369 INFO [main] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://be03.mdi.mep.ne1.yahoo.com:4080 5374 [task-runner-0] INFO io.druid.indexing.overlord.ThreadPoolTaskRunner - Running task: index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-client/0.23.10/hadoop-client-0.23.10.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-common/0.23.10/hadoop-common-0.23.10.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/google/guava/guava/11.0.2/guava-11.0.2.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/asm/asm/3.2/asm-3.2.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar] 6204 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-lang/commons-lang/2.5/commons-lang-2.5.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-auth/0.23.10/hadoop-auth-0.23.10.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/googlecode/json-simple/json-simple/1.1/json-simple-1.1.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-hdfs/0.23.10/hadoop-hdfs-0.23.10.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/0.23.10/hadoop-mapreduce-client-app-0.23.10.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/0.23.10/hadoop-mapreduce-client-common-0.23.10.jar] 6205 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/0.23.10/hadoop-yarn-server-common-0.23.10.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/zookeeper/zookeeper/3.4.2/zookeeper-3.4.2.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/google/inject/guice/3.0/guice-3.0.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-grizzly2/1.8/jersey-test-framework-grizzly2-1.8.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/javax.servlet/3.0/javax.servlet-3.0.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/sun/jersey/jersey-grizzly2/1.8/jersey-grizzly2-1.8.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/grizzly/grizzly-http/2.1.1/grizzly-http-2.1.1.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/grizzly/grizzly-framework/2.1.1/grizzly-framework-2.1.1.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/gmbal/gmbal-api-only/3.0.0-b023/gmbal-api-only-3.0.0-b023.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/external/management-api/3.0.0-b012/management-api-3.0.0-b012.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/grizzly/grizzly-http-server/2.1.1/grizzly-http-server-2.1.1.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/grizzly/grizzly-rcm/2.1.1/grizzly-rcm-2.1.1.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/grizzly/grizzly-http-servlet/2.1.1/grizzly-http-servlet-2.1.1.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/glassfish/grizzly/grizzly-framework/2.1.1/grizzly-framework-2.1.1-tests.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar] 6206 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.8/jersey-guice-1.8.jar] 6207 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/0.23.10/hadoop-mapreduce-client-shuffle-0.23.10.jar] 6207 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/jboss/netty/netty/3.2.4.Final/netty-3.2.4.Final.jar] 6207 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-yarn-api/0.23.10/hadoop-yarn-api-0.23.10.jar] 6207 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/0.23.10/hadoop-mapreduce-client-core-0.23.10.jar] 6207 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-yarn-common/0.23.10/hadoop-yarn-common-0.23.10.jar] 6207 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/0.23.10/hadoop-mapreduce-client-jobclient-0.23.10.jar] 6207 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/org/apache/hadoop/hadoop-annotations/0.23.10/hadoop-annotations-0.23.10.jar] 6207 [task-runner-0] INFO io.druid.initialization.Initialization - Added URL[file:/home/y/tmp/.m2/repository/jdiff/jdiff/1.0.9/jdiff-1.0.9.jar] SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/y/libexec/mdi_druid/lib/slf4j-simple-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/y/libexec/mdi_druid/lib/slf4j-log4j12-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/y/tmp/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory] 124 [task-runner-0] INFO io.druid.server.initialization.PropertiesModule - Loading properties from runtime.properties 2014-05-13 01:17:08,532 INFO [task-runner-0] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.0.1.Final 746 [task-runner-0] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[], localRepository='/home/y/tmp/.m2/repository', remoteRepositories=[]}] 866 [task-runner-0] INFO io.druid.initialization.Initialization - Adding local module[class io.druid.storage.hdfs.HdfsStorageDruidModule] 867 [task-runner-0] INFO io.druid.initialization.Initialization - Adding local module[class io.druid.query.aggregation.HyperloglogDruidModule] 868 [task-runner-0] INFO io.druid.initialization.Initialization - Adding local module[class com.yahoo.mobile.mdi.druid.firehose.cms.CMSDruidModule] 870 [task-runner-0] INFO io.druid.initialization.Initialization - Adding local module[class com.yahoo.druid.sketch.module.SketchAggregatorModule] 870 [task-runner-0] INFO io.druid.initialization.Initialization - Adding local module[class com.yahoo.druid.sketch.module.SketchFileStorageFirehoseModule] 1211 [task-runner-0] INFO io.druid.guice.JsonConfigurator - Loaded class[class com.metamx.emitter.core.LoggingEmitterConfig] from props[druid.emitter.logging.] as [LoggingEmitterConfig{loggerClass='com.metamx.emitter.core.LoggingEmitter', logLevel='info'}] 1285 [task-runner-0] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@3fa28ae] 1307 [task-runner-0] INFO com.metamx.metrics.SysMonitor - Loading sigar native lib at tmpPath[/tmp/yay1712802054331887705yay/libsigar-amd64-linux.so] 1311 [task-runner-0] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[class com.metamx.metrics.SysMonitor, class com.metamx.metrics.JvmMonitor]}] 1323 [task-runner-0] INFO io.druid.server.metrics.MetricsModule - Adding monitor[com.metamx.metrics.SysMonitor@58109b76] 1326 [task-runner-0] INFO io.druid.server.metrics.MetricsModule - Adding monitor[com.metamx.metrics.JvmMonitor@726be248] 1445 [task-runner-0] INFO io.druid.server.initialization.PropertiesModule - Loading properties from runtime.properties 1463 [task-runner-0] INFO io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[], localRepository='/home/y/tmp/.m2/repository', remoteRepositories=[]}] 1463 [task-runner-0] INFO io.druid.indexing.common.task.HadoopIndexTask - Starting a hadoop determine configuration job... 1710 [task-runner-0] INFO io.druid.indexer.path.StaticPathSpec - Adding paths[hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv] 1900 [task-runner-0] INFO io.druid.indexer.HadoopDruidDetermineConfigurationJob - DateTime[2014-05-01T00:00:00.000Z], spec[HadoopyShardSpec{actualSpec=NoneShardSpec, shardNum=0}] 1900 [task-runner-0] INFO io.druid.indexer.JobHelper - Deleting path[/home/y/tmp/mdi_druid/persistent/druid-indexing/flickr1_sketch/2014-05-13T011701.684Z] 8381 [task-runner-0] INFO io.druid.indexing.common.actions.RemoteTaskActionClient - Performing action for task[index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z]: LockListAction{} 8383 [task-runner-0] INFO io.druid.indexing.common.actions.RemoteTaskActionClient - Submitting action for task[index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z] to overlord[http://be03.mdi.mep.ne1.yahoo.com:4080/druid/indexer/v1/action]: LockListAction{} 8402 [task-runner-0] INFO io.druid.indexing.common.task.HadoopIndexTask - Setting version to: 2014-05-13T01:17:01.726Z 2157 [task-runner-0] INFO io.druid.indexer.HadoopDruidIndexerConfig - Running with config: { "dataSource" : "flickr1_sketch", "timestampSpec" : { "column" : "timestamp", "format" : "YYYYMMdd" }, "dataSpec" : { "format" : "tsv", "delimiter" : "\t", "columns" : [ "timestamp", "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type", "bcookieSketch" ], "dimensions" : [ "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type" ], "spatialDimensions" : [ ] }, "granularitySpec" : { "type" : "uniform", "gran" : "DAY", "intervals" : [ "2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z" ] }, "pathSpec" : { "type" : "static", "paths" : "hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv" }, "workingPath" : "/home/y/tmp/mdi_druid/persistent/druid-indexing", "segmentOutputPath" : "hdfs://be03.mdi.mep.ne1.yahoo.com:8020/projects/mdi/deepstore/flickr1_sketch", "version" : "2014-05-13T01:17:01.726Z", "partitionsSpec" : { "type" : "dimension", "partitionDimension" : null, "targetPartitionSize" : -1, "maxPartitionSize" : -1, "assumeGrouped" : false, "numShards" : -1 }, "leaveIntermediate" : false, "cleanupOnFailure" : true, "shardSpecs" : { "2014-05-01T00:00:00.000Z" : [ { "actualSpec" : { "type" : "none" }, "shardNum" : 0 } ] }, "overwriteFiles" : false, "rollupSpec" : { "aggs" : [ { "type" : "count", "name" : "count" }, { "type" : "sketchMerge", "name" : "bcookieSketch", "fieldName" : "bcookieSketch", "size" : 65536, "filter" : { "type" : "true" } } ], "rollupGranularity" : { "type" : "none" }, "rowFlushBoundary" : 500000 }, "updaterJobSpec" : null, "ignoreInvalidRows" : false } 2157 [task-runner-0] INFO io.druid.indexing.common.task.HadoopIndexTask - Starting a hadoop index generator job... 2207 [task-runner-0] INFO io.druid.indexer.path.StaticPathSpec - Adding paths[hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv] 2210 [task-runner-0] INFO io.druid.indexer.HadoopDruidIndexerJob - No updaterJobSpec set, not uploading to database 2265 [task-runner-0] INFO io.druid.indexer.path.StaticPathSpec - Adding paths[hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv] 2014-05-13 01:17:10,669 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - session.id is deprecated. Instead, use dfs.metrics.session-id 2014-05-13 01:17:10,670 INFO [task-runner-0] org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId= 2014-05-13 01:17:10,886 WARN [task-runner-0] org.apache.hadoop.mapreduce.JobSubmitter - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2014-05-13 01:17:10,887 WARN [task-runner-0] org.apache.hadoop.mapreduce.JobSubmitter - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 2014-05-13 01:17:10,919 INFO [task-runner-0] org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2014-05-13 01:17:10,940 WARN [task-runner-0] org.apache.hadoop.io.compress.snappy.LoadSnappy - Snappy native library not loaded 2014-05-13 01:17:10,954 INFO [task-runner-0] org.apache.hadoop.mapreduce.JobSubmitter - number of splits:3 2014-05-13 01:17:10,960 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 2014-05-13 01:17:10,960 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapreduce.partitioner.class is deprecated. Instead, use mapreduce.job.partitioner.class 2014-05-13 01:17:10,960 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 2014-05-13 01:17:10,960 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.output.key.comparator.class is deprecated. Instead, use mapreduce.job.output.key.comparator.class 2014-05-13 01:17:10,960 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.job.name is deprecated. Instead, use mapreduce.job.name 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.output.value.groupfn.class is deprecated. Instead, use mapreduce.job.output.group.comparator.class 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class 2014-05-13 01:17:10,961 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 2014-05-13 01:17:11,124 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-nobody/mapred/staging/nobody600207510/.staging/job_local600207510_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2014-05-13 01:17:11,125 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-nobody/mapred/staging/nobody600207510/.staging/job_local600207510_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2014-05-13 01:17:11,234 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-nobody/mapred/local/localRunner/job_local600207510_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2014-05-13 01:17:11,234 WARN [task-runner-0] org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-nobody/mapred/local/localRunner/job_local600207510_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2014-05-13 01:17:11,241 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - The url to track the job: http://localhost:8080/ 2898 [task-runner-0] INFO io.druid.indexer.IndexGeneratorJob - Job flickr1_sketch-index-generator-Optional.of([2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z]) submitted, status available at http://localhost:8080/ 2014-05-13 01:17:11,243 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Running job: job_local600207510_0001 2014-05-13 01:17:11,244 INFO [Thread-31] org.apache.hadoop.mapred.LocalJobRunner - OutputCommitter set in config null 2014-05-13 01:17:11,250 INFO [Thread-31] org.apache.hadoop.mapred.LocalJobRunner - OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter 2014-05-13 01:17:11,259 INFO [Thread-31] org.apache.hadoop.mapred.LocalJobRunner - Waiting for map tasks 2014-05-13 01:17:11,260 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - Starting task: attempt_local600207510_0001_m_000000_0 2014-05-13 01:17:11,289 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ] 2014-05-13 01:17:11,330 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584) 2014-05-13 01:17:11,331 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100 2014-05-13 01:17:11,331 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - soft limit at 83886080 2014-05-13 01:17:11,331 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600 2014-05-13 01:17:11,331 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - kvstart = 26214396; length = 6553600 3027 [LocalJobRunner Map Task Executor #0] INFO io.druid.indexer.HadoopDruidIndexerConfig - Running with config: { "dataSource" : "flickr1_sketch", "timestampSpec" : { "column" : "timestamp", "format" : "YYYYMMdd" }, "dataSpec" : { "format" : "tsv", "delimiter" : "\t", "columns" : [ "timestamp", "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type", "bcookieSketch" ], "dimensions" : [ "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type" ], "spatialDimensions" : [ ] }, "granularitySpec" : { "type" : "uniform", "gran" : "DAY", "intervals" : [ "2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z" ] }, "pathSpec" : { "type" : "static", "paths" : "hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv" }, "workingPath" : "/home/y/tmp/mdi_druid/persistent/druid-indexing", "segmentOutputPath" : "hdfs://be03.mdi.mep.ne1.yahoo.com:8020/projects/mdi/deepstore/flickr1_sketch", "version" : "2014-05-13T01:17:01.726Z", "partitionsSpec" : { "type" : "dimension", "partitionDimension" : null, "targetPartitionSize" : -1, "maxPartitionSize" : -1, "assumeGrouped" : false, "numShards" : -1 }, "leaveIntermediate" : false, "cleanupOnFailure" : true, "shardSpecs" : { "2014-05-01T00:00:00.000Z" : [ { "actualSpec" : { "type" : "none" }, "shardNum" : 0 } ] }, "overwriteFiles" : false, "rollupSpec" : { "aggs" : [ { "type" : "count", "name" : "count" }, { "type" : "sketchMerge", "name" : "bcookieSketch", "fieldName" : "bcookieSketch", "size" : 65536, "filter" : { "type" : "true" } } ], "rollupGranularity" : { "type" : "none" }, "rowFlushBoundary" : 500000 }, "updaterJobSpec" : null, "ignoreInvalidRows" : false } 2014-05-13 01:17:12,245 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Job job_local600207510_0001 running in uber mode : false 2014-05-13 01:17:12,245 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 0% reduce 0% 2014-05-13 01:17:15,435 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Spilling map output 2014-05-13 01:17:15,435 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - bufstart = 0; bufend = 77251249; bufvoid = 104857600 2014-05-13 01:17:15,435 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 24555688(98222752); length = 1658709/6553600 2014-05-13 01:17:15,435 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - (EQUATOR) 78912353 kvi 19728084(78912336) 2014-05-13 01:17:15,685 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - 2014-05-13 01:17:15,686 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Starting flush of map output 2014-05-13 01:17:16,189 INFO [SpillThread] org.apache.hadoop.mapred.MapTask - Finished spill 0 2014-05-13 01:17:16,190 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - (RESET) equator 78912353 kv 19728084(78912336) kvi 19640208(78560832) 2014-05-13 01:17:16,190 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Spilling map output 2014-05-13 01:17:16,190 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - bufstart = 78912353; bufend = 83181196; bufvoid = 104857600 2014-05-13 01:17:16,190 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - kvstart = 19728084(78912336); kvend = 19640212(78560848); length = 87873/6553600 2014-05-13 01:17:16,216 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Finished spill 1 2014-05-13 01:17:16,222 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Merger - Merging 2 sorted segments 2014-05-13 01:17:16,224 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Merger - Down to the last merge-pass, with 2 segments left of total size: 82832151 bytes 2014-05-13 01:17:17,088 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Task:attempt_local600207510_0001_m_000000_0 is done. And is in the process of committing 2014-05-13 01:17:17,094 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - map > sort 2014-05-13 01:17:17,095 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Task 'attempt_local600207510_0001_m_000000_0' done. 2014-05-13 01:17:17,095 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - Finishing task: attempt_local600207510_0001_m_000000_0 2014-05-13 01:17:17,095 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - Starting task: attempt_local600207510_0001_m_000001_0 2014-05-13 01:17:17,096 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ] 2014-05-13 01:17:17,119 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584) 2014-05-13 01:17:17,119 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100 2014-05-13 01:17:17,119 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - soft limit at 83886080 2014-05-13 01:17:17,119 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600 2014-05-13 01:17:17,119 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - kvstart = 26214396; length = 6553600 8788 [LocalJobRunner Map Task Executor #0] INFO io.druid.indexer.HadoopDruidIndexerConfig - Running with config: { "dataSource" : "flickr1_sketch", "timestampSpec" : { "column" : "timestamp", "format" : "YYYYMMdd" }, "dataSpec" : { "format" : "tsv", "delimiter" : "\t", "columns" : [ "timestamp", "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type", "bcookieSketch" ], "dimensions" : [ "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type" ], "spatialDimensions" : [ ] }, "granularitySpec" : { "type" : "uniform", "gran" : "DAY", "intervals" : [ "2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z" ] }, "pathSpec" : { "type" : "static", "paths" : "hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv" }, "workingPath" : "/home/y/tmp/mdi_druid/persistent/druid-indexing", "segmentOutputPath" : "hdfs://be03.mdi.mep.ne1.yahoo.com:8020/projects/mdi/deepstore/flickr1_sketch", "version" : "2014-05-13T01:17:01.726Z", "partitionsSpec" : { "type" : "dimension", "partitionDimension" : null, "targetPartitionSize" : -1, "maxPartitionSize" : -1, "assumeGrouped" : false, "numShards" : -1 }, "leaveIntermediate" : false, "cleanupOnFailure" : true, "shardSpecs" : { "2014-05-01T00:00:00.000Z" : [ { "actualSpec" : { "type" : "none" }, "shardNum" : 0 } ] }, "overwriteFiles" : false, "rollupSpec" : { "aggs" : [ { "type" : "count", "name" : "count" }, { "type" : "sketchMerge", "name" : "bcookieSketch", "fieldName" : "bcookieSketch", "size" : 65536, "filter" : { "type" : "true" } } ], "rollupGranularity" : { "type" : "none" }, "rowFlushBoundary" : 500000 }, "updaterJobSpec" : null, "ignoreInvalidRows" : false } 2014-05-13 01:17:17,250 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 0% 2014-05-13 01:17:20,210 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Spilling map output 2014-05-13 01:17:20,210 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - bufstart = 0; bufend = 77462423; bufvoid = 104857600 2014-05-13 01:17:20,210 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 24608452(98433808); length = 1605945/6553600 2014-05-13 01:17:20,210 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - (EQUATOR) 79075591 kvi 19768892(79075568) 2014-05-13 01:17:20,365 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - 2014-05-13 01:17:20,365 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Starting flush of map output 2014-05-13 01:17:20,711 INFO [SpillThread] org.apache.hadoop.mapred.MapTask - Finished spill 0 2014-05-13 01:17:20,711 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - (RESET) equator 79075591 kv 19768892(79075568) kvi 19691128(78764512) 2014-05-13 01:17:20,711 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Spilling map output 2014-05-13 01:17:20,711 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - bufstart = 79075591; bufend = 82615433; bufvoid = 104857600 2014-05-13 01:17:20,711 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - kvstart = 19768892(79075568); kvend = 19691132(78764528); length = 77761/6553600 2014-05-13 01:17:20,732 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Finished spill 1 2014-05-13 01:17:20,734 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Merger - Merging 2 sorted segments 2014-05-13 01:17:20,734 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Merger - Down to the last merge-pass, with 2 segments left of total size: 82267791 bytes 2014-05-13 01:17:21,336 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 33% reduce 0% 2014-05-13 01:17:21,412 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Task:attempt_local600207510_0001_m_000001_0 is done. And is in the process of committing 2014-05-13 01:17:21,414 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - map > sort 2014-05-13 01:17:21,414 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Task 'attempt_local600207510_0001_m_000001_0' done. 2014-05-13 01:17:21,414 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - Finishing task: attempt_local600207510_0001_m_000001_0 2014-05-13 01:17:21,414 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - Starting task: attempt_local600207510_0001_m_000002_0 2014-05-13 01:17:21,416 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ] 2014-05-13 01:17:21,440 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584) 2014-05-13 01:17:21,440 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100 2014-05-13 01:17:21,440 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - soft limit at 83886080 2014-05-13 01:17:21,440 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600 2014-05-13 01:17:21,440 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - kvstart = 26214396; length = 6553600 13107 [LocalJobRunner Map Task Executor #0] INFO io.druid.indexer.HadoopDruidIndexerConfig - Running with config: { "dataSource" : "flickr1_sketch", "timestampSpec" : { "column" : "timestamp", "format" : "YYYYMMdd" }, "dataSpec" : { "format" : "tsv", "delimiter" : "\t", "columns" : [ "timestamp", "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type", "bcookieSketch" ], "dimensions" : [ "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type" ], "spatialDimensions" : [ ] }, "granularitySpec" : { "type" : "uniform", "gran" : "DAY", "intervals" : [ "2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z" ] }, "pathSpec" : { "type" : "static", "paths" : "hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv" }, "workingPath" : "/home/y/tmp/mdi_druid/persistent/druid-indexing", "segmentOutputPath" : "hdfs://be03.mdi.mep.ne1.yahoo.com:8020/projects/mdi/deepstore/flickr1_sketch", "version" : "2014-05-13T01:17:01.726Z", "partitionsSpec" : { "type" : "dimension", "partitionDimension" : null, "targetPartitionSize" : -1, "maxPartitionSize" : -1, "assumeGrouped" : false, "numShards" : -1 }, "leaveIntermediate" : false, "cleanupOnFailure" : true, "shardSpecs" : { "2014-05-01T00:00:00.000Z" : [ { "actualSpec" : { "type" : "none" }, "shardNum" : 0 } ] }, "overwriteFiles" : false, "rollupSpec" : { "aggs" : [ { "type" : "count", "name" : "count" }, { "type" : "sketchMerge", "name" : "bcookieSketch", "fieldName" : "bcookieSketch", "size" : 65536, "filter" : { "type" : "true" } } ], "rollupGranularity" : { "type" : "none" }, "rowFlushBoundary" : 500000 }, "updaterJobSpec" : null, "ignoreInvalidRows" : false } 2014-05-13 01:17:21,791 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - 2014-05-13 01:17:21,791 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Starting flush of map output 2014-05-13 01:17:21,791 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Spilling map output 2014-05-13 01:17:21,791 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - bufstart = 0; bufend = 9548156; bufvoid = 104857600 2014-05-13 01:17:21,791 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26007128(104028512); length = 207269/6553600 2014-05-13 01:17:21,840 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.MapTask - Finished spill 0 2014-05-13 01:17:21,842 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Task:attempt_local600207510_0001_m_000002_0 is done. And is in the process of committing 2014-05-13 01:17:21,843 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - map 2014-05-13 01:17:21,843 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.Task - Task 'attempt_local600207510_0001_m_000002_0' done. 2014-05-13 01:17:21,843 INFO [LocalJobRunner Map Task Executor #0] org.apache.hadoop.mapred.LocalJobRunner - Finishing task: attempt_local600207510_0001_m_000002_0 2014-05-13 01:17:21,844 INFO [Thread-31] org.apache.hadoop.mapred.LocalJobRunner - Map task executor complete. 2014-05-13 01:17:21,869 INFO [Thread-31] org.apache.hadoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ] 2014-05-13 01:17:21,871 INFO [Thread-31] org.apache.hadoop.mapred.Merger - Merging 3 sorted segments 2014-05-13 01:17:21,871 INFO [Thread-31] org.apache.hadoop.mapred.Merger - Down to the last merge-pass, with 3 segments left of total size: 174804120 bytes 2014-05-13 01:17:21,871 INFO [Thread-31] org.apache.hadoop.mapred.LocalJobRunner - 2014-05-13 01:17:21,876 WARN [Thread-31] org.apache.hadoop.conf.Configuration - mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords 13544 [Thread-31] INFO io.druid.indexer.HadoopDruidIndexerConfig - Running with config: { "dataSource" : "flickr1_sketch", "timestampSpec" : { "column" : "timestamp", "format" : "YYYYMMdd" }, "dataSpec" : { "format" : "tsv", "delimiter" : "\t", "columns" : [ "timestamp", "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type", "bcookieSketch" ], "dimensions" : [ "app_id", "app_version", "event_name", "os_name", "os_ver", "woeid_country", "woeid_state", "woeid_city", "connection_speed", "birth_year", "gender", "device_type" ], "spatialDimensions" : [ ] }, "granularitySpec" : { "type" : "uniform", "gran" : "DAY", "intervals" : [ "2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z" ] }, "pathSpec" : { "type" : "static", "paths" : "hdfs://be03.mdi.mep.ne1.yahoo.com/sketches/flickr_may01.sketch.tsv" }, "workingPath" : "/home/y/tmp/mdi_druid/persistent/druid-indexing", "segmentOutputPath" : "hdfs://be03.mdi.mep.ne1.yahoo.com:8020/projects/mdi/deepstore/flickr1_sketch", "version" : "2014-05-13T01:17:01.726Z", "partitionsSpec" : { "type" : "dimension", "partitionDimension" : null, "targetPartitionSize" : -1, "maxPartitionSize" : -1, "assumeGrouped" : false, "numShards" : -1 }, "leaveIntermediate" : false, "cleanupOnFailure" : true, "shardSpecs" : { "2014-05-01T00:00:00.000Z" : [ { "actualSpec" : { "type" : "none" }, "shardNum" : 0 } ] }, "overwriteFiles" : false, "rollupSpec" : { "aggs" : [ { "type" : "count", "name" : "count" }, { "type" : "sketchMerge", "name" : "bcookieSketch", "fieldName" : "bcookieSketch", "size" : 65536, "filter" : { "type" : "true" } } ], "rollupGranularity" : { "type" : "none" }, "rowFlushBoundary" : 500000 }, "updaterJobSpec" : null, "ignoreInvalidRows" : false } 2014-05-13 01:17:22,337 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 0% 2014-05-13 01:17:27,872 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 2014-05-13 01:17:28,460 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 73% 2014-05-13 01:17:30,873 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 2014-05-13 01:17:31,462 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 78% 2014-05-13 01:17:33,873 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 2014-05-13 01:17:34,079 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 81% 27622 [Thread-31] INFO io.druid.indexer.IndexGeneratorJob - 500,000 lines to 500,000 rows in 14,071 millis 27628 [Thread-31] INFO io.druid.segment.IndexMerger - Starting persist for interval[2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z], rows[500,000] 2014-05-13 01:17:36,873 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 28735 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/index00000/v8-tmp] completed index.drd in 5 millis. 2014-05-13 01:17:37,081 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 84% 28940 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/index00000/v8-tmp] completed dim conversions in 205 millis. 29363 [Thread-31] INFO io.druid.segment.CompressedPools - Allocating new chunkEncoder[1] 2014-05-13 01:17:39,874 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 33111 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/index00000/v8-tmp] walked 500,000/500,000 rows in 4,160 millis. 33228 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/index00000/v8-tmp] completed walk through of 500,000 rows in 4,288 millis. 33230 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[app_id] with cardinality[1] 33342 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[app_id] in 114 millis. 33342 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[app_version] with cardinality[6] 33476 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[app_version] in 134 millis. 33476 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[birth_year] with cardinality[133] 33638 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[birth_year] in 162 millis. 33638 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[connection_speed] with cardinality[10] 33733 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[connection_speed] in 95 millis. 33733 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[device_type] with cardinality[2] 33781 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[device_type] in 48 millis. 33781 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[event_name] with cardinality[70] 33816 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[event_name] in 35 millis. 33816 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[gender] with cardinality[2] 33845 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[gender] in 28 millis. 33845 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[os_name] with cardinality[1] 33885 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[os_name] in 40 millis. 33885 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[os_ver] with cardinality[27] 33927 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[os_ver] in 42 millis. 33927 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_city] with cardinality[19,841] 34296 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_city] in 369 millis. 34296 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_country] with cardinality[201] 34347 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_country] in 50 millis. 34347 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_state] with cardinality[1,484] 34409 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_state] in 62 millis. 34410 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/index00000/v8-tmp] completed inverted.drd in 1,181 millis. 34506 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Converting v8[/tmp/base7640569517920786981flush/index00000/v8-tmp] to v9[/tmp/base7640569517920786981flush/index00000] 34508 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_app_id.drd] 2014-05-13 01:17:42,874 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 34551 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[app_id] is single value, converting... 34618 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_app_version.drd] 34673 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[app_version] is single value, converting... 34704 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_birth_year.drd] 34785 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[birth_year] is single value, converting... 34785 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[birth_year] has null rows. 34786 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[birth_year] has no null value in the dictionary, expanding... 34852 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_connection_speed.drd] 34920 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[connection_speed] is single value, converting... 34920 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[connection_speed] has null rows. 34920 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[connection_speed] has no null value in the dictionary, expanding... 34953 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_device_type.drd] 34968 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[device_type] is single value, converting... 34968 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[device_type] has null rows. 34968 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[device_type] has no null value in the dictionary, expanding... 34999 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_event_name.drd] 35015 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[event_name] is single value, converting... 35015 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[event_name] has null rows. 35015 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[event_name] has no null value in the dictionary, expanding... 35043 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_gender.drd] 35063 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[gender] is single value, converting... 35064 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[gender] has null rows. 35064 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[gender] has no null value in the dictionary, expanding... 35094 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_os_name.drd] 35109 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[os_name] is single value, converting... 35136 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_os_ver.drd] 35150 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[os_ver] is single value, converting... 35177 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_city.drd] 35192 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_city] is single value, converting... 35192 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_city] has null rows. 35192 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_city] has no null value in the dictionary, expanding... 35345 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_country.drd] 35360 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_country] is single value, converting... 35360 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_country] has null rows. 35360 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_country] has no null value in the dictionary, expanding... 35390 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_state.drd] 35404 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_state] is single value, converting... 35404 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_state] has null rows. 35405 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_state] has no null value in the dictionary, expanding... 35440 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[index.drd] 35440 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[inverted.drd] 35440 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[met_bcookiesketch_LITTLE_ENDIAN.drd] 35473 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[met_count_LITTLE_ENDIAN.drd] 35476 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[spatial.drd] 35476 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[time_LITTLE_ENDIAN.drd] 35479 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Skipped files[[index.drd, inverted.drd, spatial.drd]] 2014-05-13 01:17:45,875 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 2014-05-13 01:17:46,133 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 87% 2014-05-13 01:17:48,875 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 2014-05-13 01:17:49,135 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 92% 2014-05-13 01:17:51,876 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 2014-05-13 01:17:52,267 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 97% 45869 [Thread-31] INFO io.druid.indexer.IndexGeneratorJob - 909,393 lines completed. 45870 [Thread-31] INFO io.druid.segment.IndexMerger - Starting persist for interval[2014-05-01T00:00:00.000Z/2014-05-02T00:00:00.000Z], rows[409,393] 2014-05-13 01:17:54,876 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 46548 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/final/v8-tmp] completed index.drd in 1 millis. 46604 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/final/v8-tmp] completed dim conversions in 56 millis. 2014-05-13 01:17:55,268 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - map 100% reduce 100% 2014-05-13 01:17:57,877 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 50446 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/final/v8-tmp] completed walk through of 409,393 rows in 3,842 millis. 50447 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[app_id] with cardinality[1] 50565 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[app_id] in 119 millis. 50565 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[app_version] with cardinality[1] 50684 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[app_version] in 119 millis. 50684 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[birth_year] with cardinality[132] 50816 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[birth_year] in 132 millis. 50817 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[connection_speed] with cardinality[9] 50873 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[connection_speed] in 57 millis. 50873 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[device_type] with cardinality[2] 50916 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[device_type] in 43 millis. 50917 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[event_name] with cardinality[34] 50951 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[event_name] in 35 millis. 50951 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[gender] with cardinality[2] 50982 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[gender] in 31 millis. 50982 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[os_name] with cardinality[1] 51019 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[os_name] in 37 millis. 51019 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[os_ver] with cardinality[9] 51057 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[os_ver] in 38 millis. 51057 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_city] with cardinality[17,647] 51168 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_city] in 111 millis. 51168 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_country] with cardinality[196] 51209 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_country] in 41 millis. 51210 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_state] with cardinality[1,428] 51261 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_state] in 52 millis. 51261 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/final/v8-tmp] completed inverted.drd in 815 millis. 51327 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Converting v8[/tmp/base7640569517920786981flush/final/v8-tmp] to v9[/tmp/base7640569517920786981flush/final] 51327 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_app_id.drd] 51407 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[app_id] is single value, converting... 51430 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_app_version.drd] 51510 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[app_version] is single value, converting... 51533 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_birth_year.drd] 51612 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[birth_year] is single value, converting... 51613 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[birth_year] has null rows. 51613 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[birth_year] has no null value in the dictionary, expanding... 51647 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_connection_speed.drd] 51660 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[connection_speed] is single value, converting... 51682 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_device_type.drd] 51695 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[device_type] is single value, converting... 51717 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_event_name.drd] 51730 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[event_name] is single value, converting... 51730 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[event_name] has null rows. 51730 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[event_name] has no null value in the dictionary, expanding... 51752 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_gender.drd] 51769 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[gender] is single value, converting... 51769 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[gender] has null rows. 51769 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[gender] has no null value in the dictionary, expanding... 51794 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_os_name.drd] 51806 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[os_name] is single value, converting... 51829 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_os_ver.drd] 51842 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[os_ver] is single value, converting... 51865 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_city.drd] 51878 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_city] is single value, converting... 51904 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_country.drd] 51916 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_country] is single value, converting... 52152 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_state.drd] 52164 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_state] is single value, converting... 52189 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[index.drd] 52189 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[inverted.drd] 52189 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[met_bcookiesketch_LITTLE_ENDIAN.drd] 52215 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[met_count_LITTLE_ENDIAN.drd] 52216 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[spatial.drd] 52216 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[time_LITTLE_ENDIAN.drd] 52217 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Skipped files[[index.drd, inverted.drd, spatial.drd]] 52247 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/merged/v8-tmp] completed index.drd in 0 millis. 52353 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/merged/v8-tmp] completed dim conversions in 106 millis. 52360 [Thread-31] INFO io.druid.segment.CompressedPools - Allocating new littleEndByteBuf[1] 52361 [Thread-31] INFO io.druid.segment.CompressedPools - Allocating new outputBytesPool[1] 52364 [Thread-31] INFO io.druid.segment.CompressedPools - Allocating new littleEndByteBuf[2] 52365 [Thread-31] INFO io.druid.segment.CompressedPools - Allocating new littleEndByteBuf[3] 52367 [Thread-31] INFO io.druid.segment.CompressedPools - Allocating new littleEndByteBuf[4] 2014-05-13 01:18:00,877 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 2014-05-13 01:18:03,877 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 62875 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.942Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/mem/max","value":28633464832,"user1":"heap"}] 62876 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.958Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/mem/committed","value":7005536256,"user1":"heap"}] 62876 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.958Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/mem/used","value":4399391768,"user1":"heap"}] 62876 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.958Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/mem/init","value":792323264,"user1":"heap"}] 62876 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.958Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/mem/max","value":136314880,"user1":"nonheap"}] 62877 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.959Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/mem/committed","value":91947008,"user1":"nonheap"}] 62877 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.959Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/mem/used","value":66014952,"user1":"nonheap"}] 62877 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.959Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/mem/init","value":24576000,"user1":"nonheap"}] 62878 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.959Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/max","value":50331648,"user1":"nonheap","user2":"Code Cache"}] 62878 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.960Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/committed","value":5963776,"user1":"nonheap","user2":"Code Cache"}] 62878 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.960Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/used","value":5802880,"user1":"nonheap","user2":"Code Cache"}] 62878 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.960Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/init","value":2555904,"user1":"nonheap","user2":"Code Cache"}] 62879 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.960Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/max","value":10190585856,"user1":"heap","user2":"PS Eden Space"}] 62879 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.961Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/committed","value":5693767680,"user1":"heap","user2":"PS Eden Space"}] 62879 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.961Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/used","value":3521061400,"user1":"heap","user2":"PS Eden Space"}] 62880 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.961Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/init","value":198180864,"user1":"heap","user2":"PS Eden Space"}] 62880 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.962Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/max","value":243793920,"user1":"heap","user2":"PS Survivor Space"}] 62880 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.962Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/committed","value":243793920,"user1":"heap","user2":"PS Survivor Space"}] 62880 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.962Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/used","value":120157912,"user1":"heap","user2":"PS Survivor Space"}] 62881 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.962Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/init","value":33030144,"user1":"heap","user2":"PS Survivor Space"}] 62881 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.963Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/max","value":21474836480,"user1":"heap","user2":"PS Old Gen"}] 62881 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.963Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/committed","value":1067974656,"user1":"heap","user2":"PS Old Gen"}] 62881 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.963Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/used","value":758172456,"user1":"heap","user2":"PS Old Gen"}] 62882 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.963Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/init","value":527958016,"user1":"heap","user2":"PS Old Gen"}] 62882 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.964Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/max","value":85983232,"user1":"nonheap","user2":"PS Perm Gen"}] 62882 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.964Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/committed","value":85983232,"user1":"nonheap","user2":"PS Perm Gen"}] 62882 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.964Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/used","value":60316312,"user1":"nonheap","user2":"PS Perm Gen"}] 62883 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.964Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/pool/init","value":22020096,"user1":"nonheap","user2":"PS Perm Gen"}] 62883 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.965Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/bufferpool/capacity","value":3659171,"user2":"direct"}] 62883 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.965Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/bufferpool/used","value":3659171,"user2":"direct"}] 62884 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.965Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/bufferpool/count","value":83,"user2":"direct"}] 62884 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.966Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/bufferpool/capacity","value":105525715,"user2":"mapped"}] 62884 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.966Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/bufferpool/used","value":105525715,"user2":"mapped"}] 62884 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.966Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"jvm/bufferpool/count","value":3,"user2":"mapped"}] 62886 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.968Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/mem/max","value":50708688896}] 62887 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.968Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/mem/used","value":19985018880}] 62891 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.971Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/max","value":10568916992,"user1":"/dev/mapper/sys-root","user2":"/","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62891 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.973Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/used","value":3425873920,"user1":"/dev/mapper/sys-root","user2":"/","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62892 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.973Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/max","value":495499264,"user1":"/dev/sda1","user2":"/boot","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62892 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.974Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/used","value":83560448,"user1":"/dev/sda1","user2":"/boot","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62893 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.974Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/max","value":460044632064,"user1":"/dev/mapper/sys-home","user2":"/home","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62893 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.975Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/used","value":9080242176,"user1":"/dev/mapper/sys-home","user2":"/home","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62894 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.975Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/max","value":4227530752,"user1":"/dev/mapper/sys-tmp","user2":"/tmp","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62894 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.976Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/used","value":781545472,"user1":"/dev/mapper/sys-tmp","user2":"/tmp","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62894 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.976Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/max","value":4227530752,"user1":"/dev/mapper/sys-var","user2":"/var","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62895 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.977Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/fs/used","value":1100591104,"user1":"/dev/mapper/sys-var","user2":"/var","user3":"local","user4":"ext4","user5":["rw","noatime"]}] 62902 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.984Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/swap/pageIn","value":0}] 62902 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.984Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/swap/pageOut","value":0}] 62902 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.984Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/swap/max","value":12884893696}] 62902 [MonitorScheduler-0] INFO com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"metrics","timestamp":"2014-05-13T01:18:04.984Z","service":"overlord","host":"be03.mdi.mep.ne1.yahoo.com:8085","metric":"sys/swap/free","value":12884893696}] 57113 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/merged/v8-tmp] walked 500,000/500,000 rows in 4,755 millis. 2014-05-13 01:18:06,878 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 60609 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/merged/v8-tmp] completed walk through of 909,393 rows in 8,256 millis. 60609 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[app_id] with cardinality[1] 60737 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[app_id] in 128 millis. 60738 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[app_version] with cardinality[6] 61032 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[app_version] in 295 millis. 61032 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[birth_year] with cardinality[134] 61437 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[birth_year] in 405 millis. 61437 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[connection_speed] with cardinality[11] 2014-05-13 01:18:09,878 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 61749 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[connection_speed] in 312 millis. 61750 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[device_type] with cardinality[3] 62071 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[device_type] in 322 millis. 62071 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[event_name] with cardinality[78] 62358 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[event_name] in 287 millis. 62358 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[gender] with cardinality[3] 62615 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[gender] in 257 millis. 62616 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[os_name] with cardinality[1] 62721 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[os_name] in 106 millis. 62721 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[os_ver] with cardinality[27] 62827 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[os_ver] in 106 millis. 62827 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_city] with cardinality[19,861] 63282 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_city] in 455 millis. 63282 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_country] with cardinality[202] 63404 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_country] in 122 millis. 63405 [Thread-31] INFO io.druid.segment.IndexMerger - Starting dimension[woeid_state] with cardinality[1,487] 63559 [Thread-31] INFO io.druid.segment.IndexMerger - Completed dimension[woeid_state] in 154 millis. 63559 [Thread-31] INFO io.druid.segment.IndexMerger - outDir[/tmp/base7640569517920786981flush/merged/v8-tmp] completed inverted.drd in 2,950 millis. 63704 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Converting v8[/tmp/base7640569517920786981flush/merged/v8-tmp] to v9[/tmp/base7640569517920786981flush/merged] 63705 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_app_id.drd] 63734 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[app_id] is single value, converting... 63786 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_app_version.drd] 63813 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[app_version] is single value, converting... 63865 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_birth_year.drd] 63892 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[birth_year] is single value, converting... 63948 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_connection_speed.drd] 63975 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[connection_speed] is single value, converting... 64027 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_device_type.drd] 64054 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[device_type] is single value, converting... 64103 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_event_name.drd] 64129 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[event_name] is single value, converting... 64178 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_gender.drd] 64204 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[gender] is single value, converting... 64254 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_os_name.drd] 64280 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[os_name] is single value, converting... 64328 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_os_ver.drd] 64355 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[os_ver] is single value, converting... 64404 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_city.drd] 64430 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_city] is single value, converting... 64484 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_country.drd] 64511 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_country] is single value, converting... 2014-05-13 01:18:12,879 INFO [communication thread] org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce 64560 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[dim_woeid_state.drd] 64587 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Dimension[woeid_state] is single value, converting... 64643 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[index.drd] 64643 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[inverted.drd] 64643 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[met_bcookiesketch_LITTLE_ENDIAN.drd] 64698 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[met_count_LITTLE_ENDIAN.drd] 64698 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[spatial.drd] 64699 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Processing file[time_LITTLE_ENDIAN.drd] 64700 [Thread-31] INFO io.druid.segment.IndexIO$DefaultIndexIOHandler - Skipped files[[index.drd, inverted.drd, spatial.drd]] 2014-05-13 01:18:13,051 WARN [Thread-31] org.apache.hadoop.mapred.LocalJobRunner - job_local600207510_0001 java.lang.IllegalArgumentException: Pathname /projects/mdi/deepstore/flickr1_sketch/flickr1_sketch/2014-05-01T00:00:00.000Z_2014-05-02T00:00:00.000Z/2014-05-13T01:17:01.726Z/0 from hdfs://be03.mdi.mep.ne1.yahoo.com:8020/projects/mdi/deepstore/flickr1_sketch/flickr1_sketch/2014-05-01T00:00:00.000Z_2014-05-02T00:00:00.000Z/2014-05-13T01:17:01.726Z/0 is not a valid DFS filename. at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:158) at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:479) at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1807) at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.serializeOutIndex(IndexGeneratorJob.java:404) at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:384) at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:247) at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:170) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:636) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:396) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:441) Exception in thread "Thread-31" java.lang.NoClassDefFoundError: org/apache/commons/httpclient/HttpMethod at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:480) Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.HttpMethod at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 1 more 2014-05-13 01:18:13,470 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Job job_local600207510_0001 failed with state FAILED due to: NA 2014-05-13 01:18:13,487 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Counters: 32 File System Counters FILE: Number of bytes read=752940904 FILE: Number of bytes written=1176280894 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=485466248 HDFS: Number of bytes written=0 HDFS: Number of read operations=17 HDFS: Number of large read operations=0 HDFS: Number of write operations=0 Map-Reduce Framework Map input records=909393 Map output records=909393 Map output bytes=172070513 Map output materialized bytes=174804225 Input split bytes=393 Combine input records=0 Combine output records=0 Reduce input groups=1 Reduce shuffle bytes=0 Reduce input records=909393 Reduce output records=0 Spilled Records=2676361 Shuffled Maps =0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=2616 CPU time spent (ms)=0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Total committed heap usage (bytes)=13411287040 File Input Format Counters Bytes Read=142063684 File Output Format Counters Bytes Written=8 65154 [task-runner-0] INFO io.druid.indexer.JobHelper - Deleting path[/home/y/tmp/mdi_druid/persistent/druid-indexing/flickr1_sketch/2014-05-13T011701.726Z] 71431 [task-runner-0] ERROR io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z, type=index_hadoop, dataSource=flickr1_sketch}] java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:221) at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:224) at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:203) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: com.metamx.common.ISE: Job[class io.druid.indexer.IndexGeneratorJob] failed! at io.druid.indexer.JobHelper.runJobs(JobHelper.java:154) at io.druid.indexer.HadoopDruidIndexerJob.run(HadoopDruidIndexerJob.java:80) at io.druid.indexing.common.task.HadoopIndexTask$HadoopIndexGeneratorInnerProcessing.runTask(HadoopIndexTask.java:260) ... 11 more 71436 [task-runner-0] INFO io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: { "id" : "index_hadoop_flickr1_sketch_2014-05-13T01:17:01.715Z", "status" : "FAILED", "duration" : 66058 } 71441 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.server.coordination.AbstractDataSegmentAnnouncer.stop()] on object[io.druid.server.coordination.BatchDataSegmentAnnouncer@5692fc6]. 71441 [main] INFO io.druid.server.coordination.AbstractDataSegmentAnnouncer - Stopping class io.druid.server.coordination.BatchDataSegmentAnnouncer with config[io.druid.server.initialization.ZkPathsConfig$$EnhancerByCGLIB$$3ec84255@3f6108bc] 71441 [main] INFO io.druid.curator.announcement.Announcer - unannouncing [/druid/announcements/be03.mdi.mep.ne1.yahoo.com:8085] 71460 [ServerInventoryView-0] INFO io.druid.curator.inventory.CuratorInventoryManager - Closing inventory cache for be03.mdi.mep.ne1.yahoo.com:8085. Also removing listeners. 71460 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.indexing.worker.executor.ExecutorLifecycle.stop()] on object[io.druid.indexing.worker.executor.ExecutorLifecycle@44f0adaf]. 71460 [ServerInventoryView-0] INFO io.druid.client.BatchServerInventoryView - Server Disappeared[DruidServerMetadata{name='be03.mdi.mep.ne1.yahoo.com:8085', host='be03.mdi.mep.ne1.yahoo.com:8085', maxSize=0, tier='_default_tier', type='indexer-executor', priority='0'}] 71468 [main] INFO org.eclipse.jetty.server.ServerConnector - Stopped ServerConnector@3562626{HTTP/1.1}{0.0.0.0:8085} 71471 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.ServletContextHandler@8ddadc8{/,null,UNAVAILABLE} 71477 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.indexing.overlord.ThreadPoolTaskRunner.stop()] on object[io.druid.indexing.overlord.ThreadPoolTaskRunner@325adead]. 71478 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.client.ServerInventoryView.stop() throws java.io.IOException] on object[io.druid.client.BatchServerInventoryView@6504e8ae]. 71479 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.curator.announcement.Announcer.stop()] on object[io.druid.curator.announcement.Announcer@55fa81]. 71480 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.curator.discovery.ServerDiscoverySelector.stop() throws java.io.IOException] on object[io.druid.curator.discovery.ServerDiscoverySelector@5b93ceb6]. 71482 [main] INFO io.druid.curator.CuratorModule - Stopping Curator 71491 [main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x345d566ea5f0020 closed 71491 [main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down 71491 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.http.client.HttpClient.stop()] on object[com.metamx.http.client.HttpClient@5d2c8a22]. 71584 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.metrics.MonitorScheduler.stop()] on object[com.metamx.metrics.MonitorScheduler@67d3f1cb]. 71584 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.emitter.service.ServiceEmitter.close() throws java.io.IOException] on object[com.metamx.emitter.service.ServiceEmitter@58849313]. 71584 [main] INFO com.metamx.emitter.core.LoggingEmitter - Close: started [false] 71585 [main] INFO com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.emitter.core.LoggingEmitter.close() throws java.io.IOException] on object[com.metamx.emitter.core.LoggingEmitter@73b0a5b].
Nishant Bangarwa
May 14, 2014, 5:42:14 AM5/14/14
to druid-de...@googlegroups.com
Hi Michael,
Druid checks the default file system while replacing ":",
What is the value of fs.defaultFS set in hadoop config files ?
can you try pointing this to hdfs filesystem, If its not already doing that ?
--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/b8fc2eb3-e58f-42cc-a38f-523d28403176%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
|
Reply all
Reply to author
Forward
0 new messages