Error while sending data to druid through tranqulity

907 views
Skip to first unread message

Ritesh Sinha

unread,
Dec 16, 2015, 4:07:25 AM12/16/15
to druid...@googlegroups.com
Hi,

I am learning druid. I am able to insert and query data through command line.Now, I am trying to insert data using tranquility.I see we use overlord and middle manager instead of realtime node. I got a code, but it throws the following error.I am not very sure how to handle this.Any insight will be helpful.




Exception in thread "main" java.lang.NoSuchMethodError: scala.util.matching.Regex.unapplySeq(Ljava/lang/CharSequence;)Lscala/Option;
at com.fasterxml.jackson.module.scala.JacksonModule$.version$lzycompute(JacksonModule.scala:30)
at com.fasterxml.jackson.module.scala.JacksonModule$.version(JacksonModule.scala:26)
at com.fasterxml.jackson.module.scala.JacksonModule$class.version(JacksonModule.scala:49)
at com.fasterxml.jackson.module.scala.DefaultScalaModule.version(DefaultScalaModule.scala:19)
at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:701)
at com.metamx.common.scala.Jackson$$anonfun$newObjectMapper$1.apply(Jackson.scala:70)
at com.metamx.common.scala.Jackson$$anonfun$newObjectMapper$1.apply(Jackson.scala:68)
at com.metamx.common.scala.Predef$EffectOps.withEffect(Predef.scala:44)
at com.metamx.common.scala.Jackson$class.newObjectMapper(Jackson.scala:67)
at com.metamx.common.scala.Jackson$.newObjectMapper(Jackson.scala:10)
at com.metamx.common.scala.Jackson$class.newObjectMapper(Jackson.scala:64)
at com.metamx.common.scala.Jackson$.newObjectMapper(Jackson.scala:10)
at com.metamx.common.scala.Jackson$class.$init$(Jackson.scala:14)
at com.metamx.common.scala.Jackson$.<init>(Jackson.scala:10)
at com.metamx.common.scala.Jackson$.<clinit>(Jackson.scala)
at com.metamx.tranquility.druid.DruidBeams$BuilderConfig$$anon$6.<init>(DruidBeams.scala:260)
at com.metamx.tranquility.druid.DruidBeams$BuilderConfig.buildAll(DruidBeams.scala:259)
at com.metamx.tranquility.druid.DruidBeams$Builder.buildBeam(DruidBeams.scala:182)
at com.metamx.tranquility.druid.DruidBeams$Builder.buildService(DruidBeams.scala:228)
at com.metamx.tranquility.druid.DruidBeams$Builder.buildJavaService(DruidBeams.scala:232)
at MyTranquilityTest.main(MyTranquilityTest.java:84)




This is the dependency i am using :

                <dependency>
<groupId>io.druid</groupId>
<artifactId>tranquility-core_2.11</artifactId>
<version>0.6.4</version>
</dependency>

I have started coordinator,broker, historical , overlord and middle manager node.Do, i need to do some other steps as well?
Do i need to have mysql ?


This is the code :

import java.io.IOException;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;

import com.google.common.collect.ImmutableList;
import com.metamx.common.Granularity;
import com.metamx.tranquility.beam.ClusteredBeamTuning;
import com.metamx.tranquility.druid.*;
import com.metamx.tranquility.typeclass.Timestamper;
import com.twitter.finagle.Service;
import com.twitter.util.Await;
import com.twitter.util.Future;

import io.druid.granularity.QueryGranularity;
import io.druid.query.aggregation.AggregatorFactory;
import io.druid.query.aggregation.CountAggregatorFactory;
import io.druid.query.aggregation.DoubleSumAggregatorFactory;

import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.retry.ExponentialBackoffRetry;
import org.codehaus.jackson.map.JsonMappingException;
import org.codehaus.jackson.map.ObjectMapper;
import org.joda.time.DateTime;
import org.joda.time.Period;

public class MyTranquilityTest {
final static String indexService = "overlord"; 
final static String firehosePattern = "firehose:%s"; 
final static String discoveryPath = "/discovery"; 
final static String dataSource = "tranq_test";
final static List<String> dimensions = ImmutableList.of("campaignid",
"region");
final static List<AggregatorFactory> aggregators = ImmutableList.of(
new CountAggregatorFactory("count"),
new DoubleSumAggregatorFactory("impressions", "impressions"),
new DoubleSumAggregatorFactory("clicks", "clicks"));

@SuppressWarnings("serial")
final static Timestamper<Map<String, Object>> timestamper = new Timestamper<Map<String, Object>>() {
@Override
public DateTime timestamp(Map<String, Object> theMap) {
return new DateTime(theMap.get("date"));
}
};

public static void main(String[] args) {
System.out.println("Start of Tranquility");
try {
final CuratorFramework curator = CuratorFrameworkFactory.builder()
.connectString("localhost:2181")
.retryPolicy(new ExponentialBackoffRetry(1000, 20, 30000))
.build();
curator.start();

final Service<List<Map<String, Object>>, Integer> druidService = DruidBeams
.builder(timestamper)
.curator(curator)
.discoveryPath(discoveryPath)
.location(
DruidLocation.create(indexService, firehosePattern,
dataSource))
.rollup(DruidRollup.create(
DruidDimensions.specific(dimensions), aggregators,
QueryGranularity.DAY))
.tuning(ClusteredBeamTuning.create(Granularity.DAY,
new Period("PT0.5S"), new Period("P1D"), 1, 1))
.buildJavaService();
String json = "[{ \"campaignid\":\"1\", \"date\":\"2014-10-01\", \"region\":\"us\", \"impressions\":100, \"clicks\":10 }, { \"campaignid\":\"1\", \"date\":\"2014-10-02\", \"region\":\"us\", \"impressions\":100, \"clicks\":9 }, { \"campaignid\":\"2\", \"date\":\"2014-10-03\", \"region\":\"uk\", \"impressions\":110, \"clicks\":1 }, { \"campaignid\":\"3\", \"date\":\"2014-10-02\", \"region\":\"ca\", \"impressions\":100, \"clicks\":20 }] ";
List<Map<String, Object>> value = convertAJsonToAListMap(json);
System.out.println("value:" + value);
// Send events to Druid:
final Future<Integer> numSentFuture = druidService.apply(value);
// Wait for confirmation:
final Integer numSent = Await.result(numSentFuture);
System.out.println("numSent:" + numSent + ", numSentFuture:"
+ numSentFuture);
// Close lifecycled objects:
Await.result(druidService.close());
curator.close();
System.out.println("End of Tranquility");

} catch (Exception e) {
// Log
System.out.println("Message:" + e.getMessage());
}
}

@SuppressWarnings("unchecked")
public static List<Map<String, Object>> convertAJsonToAListMap(String json) {
List<Map<String, Object>> list = new LinkedList<Map<String, Object>>();
List<Map<String, Object>> listSample = new LinkedList<Map<String, Object>>();
ObjectMapper mapper = new ObjectMapper();
try {
list = mapper.readValue(json, listSample.getClass());
} catch (JsonMappingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}

return list;
}
}


overlord property :

druid.host=localhost
druid.port=8090
druid.service=overlord
druid.discovery.curator.path=/discovery


druid.indexer.queue.startDelay=PT0M

druid.indexer.runner.javaOpts=-server -Xmx256m

druid.indexer.fork.property.druid.processing.numThreads=1
druid.indexer.fork.property.druid.computation.buffer.size=100000000


 druid.indexer.runner.type=remote


middle manager property :

druid.host=localhost
druid.service=middleManager
druid.port=8100

# peon configuration
druid.indexer.fork.property.druid.processing.buffer.sizeBytes=536870912
druid.indexer.fork.property.druid.processing.numThreads=2
druid.indexer.fork.property.druid.segmentCache.locations=[{"path": "/tmp/persistent/zk_druid", "maxSize": 0}]
druid.indexer.fork.property.druid.server.http.numThreads=50

# Resources for peons
druid.indexer.runner.javaOpts=-server -Xmx3g -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
druid.indexer.task.baseTaskDir=/tmp/persistent/task/

# druid.worker.capacity=default (Number of available processors - 1)
druid.worker.ip=localhost





Ritesh Sinha

unread,
Dec 16, 2015, 7:31:31 AM12/16/15
to Druid User
I am able to fix the error.But the data does not get inserted into druid.It doesn't throw any error as well.

Ritesh Sinha

unread,
Dec 16, 2015, 7:36:38 AM12/16/15
to Druid User
This is the result i am getting when i print numset and numsentfuture.



final Future<Integer> numSentFuture = druidService.apply(value);
// Wait for confirmation:
final Integer numSent = Await.result(numSentFuture);
System.out.println("numSent:" + numSent + ", numSentFuture:"
+ numSentFuture);


numSent:0, numSentFuture:Promise@1494089143(state=Done(Return(0)))

Can anyone point what i am missing.

David Lim

unread,
Dec 16, 2015, 1:17:23 PM12/16/15
to Druid User
Hey Ritesh,

Could you post the full Tranquility logs?

Also, do you have a common.runtime.properties file loaded by the middle manager? Make sure that you set 'druid.selectors.indexing.serviceName=overlord' since you've changed the default name of druid.service on the overlord.

Ritesh Sinha

unread,
Dec 17, 2015, 1:09:57 AM12/17/15
to Druid User
Hi David ,

Thanks for replying.

druid.selectors.indexing.serviceName is set to overlord.



This is my common.runtime.properties. 
---------------------------------------------------------------------------------------------------------------------------------------------------------
#
# Druid - a distributed column store.
# Copyright 2012 - 2015 Metamarkets Group Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# Extensions (no deep storage model is listed - using local fs for deep storage - not recommended for production)
# Also, for production to use mysql add, "io.druid.extensions:mysql-metadata-storage"
druid.extensions.coordinates=["io.druid.extensions:druid-examples","io.druid.extensions:druid-kafka-eight"]
druid.extensions.localRepository=extensions-repo

# Zookeeper
druid.zk.service.host=localhost

# Metadata Storage (use something like mysql in production by uncommenting properties below)
# by default druid will use derby
# druid.metadata.storage.type=mysql
# druid.metadata.storage.connector.connectURI=jdbc:mysql://localhost:3306/druid
# druid.metadata.storage.connector.user=druid
# druid.metadata.storage.connector.password=diurd

# Deep storage (local filesystem for examples - don't use this in production)
druid.storage.type=local
druid.storage.storageDirectory=/home/ritesh/druid-0.8.2/localstorage

# Query Cache (we use a simple 10mb heap-based local cache on the broker)
druid.cache.type=local
druid.cache.sizeInBytes=10000000

# Indexing service discovery
druid.selectors.indexing.serviceName=overlord

# Monitoring (disabled for examples, if you enable SysMonitor, make sure to include sigar jar in your cp)
# druid.monitoring.monitors=["com.metamx.metrics.SysMonitor","com.metamx.metrics.JvmMonitor"]

# Metrics logging (disabled for examples - change this to logging or http in production)
druid.emitter=noop

--------------------------------------------------------------------------------------------------------------------------------------------------------------

I am not sure how to load common.runtime.properties file by the middle manager.

I am using these commands: 

#To start the coordinator node:
nohup sudo java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath config/_common:config/coordinator:lib/* io.druid.cli.Main server coordinator &

#To start the historical node:
nohup sudo java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath config/_common:config/historical:lib/* io.druid.cli.Main server historical &

#To start the broker node:
nohup java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath config/_common:config/broker:lib/* io.druid.cli.Main server broker &



#To start the overlord node:
java -Xmx500m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/overlord:config/_common io.druid.cli.Main server overlord

#To start the middlemanager node :
java -Xmx500m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/middleManager:config/_common io.druid.cli.Main server middleManager

SO, i guess common.runtime.properties file is loaded from the midlemanager.

Also , I don't see any log for Tranquility, I am posting the log which is generated while starting overlord.

2015-12-17T05:44:56,612 INFO [main] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties
2015-12-17T05:44:56,615 INFO [main] io.druid.guice.PropertiesModule - Loading properties from runtime.properties
Dec 17, 2015 5:44:56 AM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 5.1.3.Final
2015-12-17T05:44:57,086 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[io.druid.extensions:druid-examples, io.druid.extensions:druid-kafka-eight], defaultVersion='0.8.2', localRepository='extensions-repo', remoteRepositories=[https://repo1.maven.org/maven2/, https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local]}]
2015-12-17T05:44:57,224 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-examples] for class[io.druid.cli.CliCommandCreator]
2015-12-17T05:44:58,342 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/io/druid/extensions/druid-examples/0.8.2/druid-examples-0.8.2.jar]
2015-12-17T05:44:58,342 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/twitter4j/twitter4j-core/3.0.3/twitter4j-core-3.0.3.jar]
2015-12-17T05:44:58,343 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/twitter4j/twitter4j-async/3.0.3/twitter4j-async-3.0.3.jar]
2015-12-17T05:44:58,343 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/twitter4j/twitter4j-stream/3.0.3/twitter4j-stream-3.0.3.jar]
2015-12-17T05:44:58,343 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/commons-validator/commons-validator/1.4.0/commons-validator-1.4.0.jar]
2015-12-17T05:44:58,343 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/commons-beanutils/commons-beanutils/1.8.3/commons-beanutils-1.8.3.jar]
2015-12-17T05:44:58,343 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/commons-digester/commons-digester/1.8/commons-digester-1.8.jar]
2015-12-17T05:44:58,343 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar]
2015-12-17T05:44:58,345 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-kafka-eight] for class[io.druid.cli.CliCommandCreator]
2015-12-17T05:44:58,475 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/io/druid/extensions/druid-kafka-eight/0.8.2/druid-kafka-eight-0.8.2.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/apache/kafka/kafka_2.10/0.8.2.1/kafka_2.10-0.8.2.1.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/com/yammer/metrics/metrics-core/2.2.0/metrics-core-2.2.0.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/apache/kafka/kafka-clients/0.8.2.1/kafka-clients-0.8.2.1.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/log4j/log4j/1.2.16/log4j-1.2.16.jar]
2015-12-17T05:44:58,476 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/jline/jline/0.9.94/jline-0.9.94.jar]
2015-12-17T05:44:58,477 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/io/netty/netty/3.7.0.Final/netty-3.7.0.Final.jar]
2015-12-17T05:44:58,477 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar]
2015-12-17T05:44:58,477 INFO [main] io.druid.initialization.Initialization - Added URL[file:/home/ritesh/druid-0.8.2/extensions-repo/com/101tec/zkclient/0.3/zkclient-0.3.jar]
2015-12-17T05:44:58,620 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-examples] for class[io.druid.initialization.DruidModule]
2015-12-17T05:44:58,621 INFO [main] io.druid.initialization.Initialization - Adding remote extension module[io.druid.examples.ExamplesDruidModule] for class[io.druid.initialization.DruidModule]
2015-12-17T05:44:58,621 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-kafka-eight] for class[io.druid.initialization.DruidModule]
2015-12-17T05:44:58,623 INFO [main] io.druid.initialization.Initialization - Adding remote extension module[io.druid.firehose.kafka.KafkaEightDruidModule] for class[io.druid.initialization.DruidModule]
2015-12-17T05:44:59,269 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.metadata.MetadataStorageConnectorConfig] from props[druid.metadata.storage.connector.] as [DbConnectorConfig{createTables=true, connectURI='jdbc:derby://localhost:1527/druid;create=true', user='null', passwordProvider=null}]
2015-12-17T05:44:59,281 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.metadata.MetadataStorageTablesConfig] from props[druid.metadata.storage.tables.] as [io.druid.metadata.MetadataStorageTablesConfig@46bc7325]
2015-12-17T05:44:59,312 INFO [main] io.druid.metadata.storage.derby.DerbyConnector - Configured Derby as metadata storage
2015-12-17T05:44:59,353 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.common.config.ConfigManagerConfig] from props[druid.manager.config.] as [io.druid.common.config.ConfigManagerConfig@62832aa3]
2015-12-17T05:44:59,362 INFO [main] io.druid.metadata.storage.derby.DerbyConnector - Configured Derby as metadata storage
2015-12-17T05:44:59,376 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.DruidNode] from props[druid.] as [DruidNode{serviceName='overlord', host='localhost', port=8090}]
2015-12-17T05:44:59,379 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.audit.SQLAuditManagerConfig] from props[druid.audit.manager.] as [io.druid.server.audit.SQLAuditManagerConfig@17d39634]
2015-12-17T05:44:59,389 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@6d497031]
2015-12-17T05:44:59,399 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[]}]
2015-12-17T05:44:59,410 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ServerConfig] from props[druid.server.http.] as [ServerConfig{numThreads=40, maxIdleTime=PT5M}]
2015-12-17T05:44:59,416 INFO [main] org.eclipse.jetty.util.log - Logging initialized @3393ms
2015-12-17T05:44:59,491 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.config.TaskQueueConfig] from props[druid.indexer.queue.] as [io.druid.indexing.overlord.config.TaskQueueConfig@3ffc6ef7]
2015-12-17T05:44:59,498 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.TaskStorageConfig] from props[druid.indexer.storage.] as [io.druid.indexing.common.config.TaskStorageConfig@53307de3]
2015-12-17T05:44:59,514 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ZkPathsConfig] from props[druid.zk.paths.] as [io.druid.server.initialization.ZkPathsConfig@58d3f4be]
2015-12-17T05:44:59,517 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.IndexerZkConfig] from props[druid.zk.paths.indexer.] as [io.druid.server.initialization.IndexerZkConfig@d5bd5202]
2015-12-17T05:44:59,522 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.curator.CuratorConfig] from props[druid.zk.service.] as [io.druid.curator.CuratorConfig@260bbaf0]
2015-12-17T05:44:59,528 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29
2015-12-17T05:44:59,576 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.config.RemoteTaskRunnerConfig] from props[druid.indexer.runner.] as [io.druid.indexing.overlord.config.RemoteTaskRunnerConfig@56c8ca34]
2015-12-17T05:44:59,582 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.http.DruidHttpClientConfig] from props[druid.global.http.] as [io.druid.guice.http.DruidHttpClientConfig@639ce63d]
2015-12-17T05:44:59,631 INFO [main] io.druid.common.config.ConfigManager - Creating watch for key[worker.config]
2015-12-17T05:44:59,649 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.autoscaling.SimpleResourceManagementConfig] from props[druid.indexer.autoscale.] as [io.druid.indexing.overlord.autoscaling.SimpleResourceManagementConfig@dfa9549]
2015-12-17T05:44:59,679 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.autoscaling.ResourceManagementSchedulerConfig] from props[druid.indexer.autoscale.] as [io.druid.indexing.overlord.autoscaling.ResourceManagementSchedulerConfig@80b1d65]
2015-12-17T05:44:59,681 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.CuratorDiscoveryConfig] from props[druid.discovery.curator.] as [io.druid.server.initialization.CuratorDiscoveryConfig@41de3257]
2015-12-17T05:44:59,931 INFO [main] io.druid.metadata.SQLMetadataConnector - Table[druid_config] already exists
2015-12-17T05:44:59,933 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.common.config.ConfigManager.start()] on object[io.druid.common.config.ConfigManager@53ec7c54].
2015-12-17T05:44:59,935 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.emitter.service.ServiceEmitter.start()] on object[com.metamx.emitter.service.ServiceEmitter@118c8f09].
2015-12-17T05:44:59,957 INFO [main] io.druid.metadata.SQLMetadataConnector - Table[druid_audit] already exists
2015-12-17T05:44:59,958 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.metrics.MonitorScheduler.start()] on object[com.metamx.metrics.MonitorScheduler@58fcdb79].
2015-12-17T05:44:59,959 INFO [main] io.druid.curator.CuratorModule - Starting Curator
2015-12-17T05:44:59,959 INFO [main] org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
2015-12-17T05:44:59,969 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-12-17T05:44:59,969 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:host.name=ritesh
2015-12-17T05:44:59,969 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.7.0_91
2015-12-17T05:44:59,969 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=lib/jackson-jaxrs-smile-provider-2.4.6.jar:lib/jetty-security-9.2.5.v20141112.jar:lib/log4j-1.2-api-2.3.jar:lib/wagon-provider-api-2.4.jar:lib/jackson-mapper-asl-1.9.13.jar:lib/aws-java-sdk-codepipeline-1.10.21.jar:lib/curator-recipes-2.8.0.jar:lib/aws-java-sdk-rds-1.10.21.jar:lib/jackson-jaxrs-base-2.4.6.jar:lib/maven-settings-builder-3.1.1.jar:lib/druid-common-0.8.2.jar:lib/aws-java-sdk-glacier-1.10.21.jar:lib/aws-java-sdk-elastictranscoder-1.10.21.jar:lib/aws-java-sdk-cognitoidentity-1.10.21.jar:lib/aws-java-sdk-elasticache-1.10.21.jar:lib/jdbi-2.63.1.jar:lib/guava-16.0.1.jar:lib/bytebuffer-collections-0.2.1.jar:lib/commons-dbcp2-2.0.1.jar:lib/jackson-datatype-joda-2.4.6.jar:lib/antlr4-runtime-4.0.jar:lib/aws-java-sdk-sns-1.10.21.jar:lib/aws-java-sdk-directconnect-1.10.21.jar:lib/joda-time-2.8.2.jar:lib/aws-java-sdk-simpleworkflow-1.10.21.jar:lib/rhino-1.7R5.jar:lib/aws-java-sdk-cloudwatchmetrics-1.10.21.jar:lib/druid-indexing-hadoop-0.8.2.jar:lib/server-metrics-0.2.5.jar:lib/jersey-core-1.19.jar:lib/opencsv-2.3.jar:lib/jetty-client-9.2.5.v20141112.jar:lib/log4j-slf4j-impl-2.3.jar:lib/commons-pool2-2.2.jar:lib/aws-java-sdk-support-1.10.21.jar:lib/disruptor-3.3.0.jar:lib/jsr305-2.0.1.jar:lib/spymemcached-2.11.7.jar:lib/aws-java-sdk-simpledb-1.10.21.jar:lib/jetty-util-9.2.5.v20141112.jar:lib/aopalliance-1.0.jar:lib/http-client-1.0.4.jar:lib/validation-api-1.1.0.Final.jar:lib/aws-java-sdk-datapipeline-1.10.21.jar:lib/jline-0.9.94.jar:lib/jetty-proxy-9.2.5.v20141112.jar:lib/aws-java-sdk-importexport-1.10.21.jar:lib/jackson-dataformat-smile-2.4.6.jar:lib/classmate-1.0.0.jar:lib/aether-connector-okhttp-0.0.9.jar:lib/derbynet-10.11.1.1.jar:lib/aws-java-sdk-elasticbeanstalk-1.10.21.jar:lib/aws-java-sdk-sqs-1.10.21.jar:lib/jackson-core-asl-1.9.13.jar:lib/maven-settings-3.1.1.jar:lib/jetty-servlets-9.2.5.v20141112.jar:lib/maxminddb-0.2.0.jar:lib/druid-api-0.3.13.jar:lib/slf4j-api-1.6.4.jar:lib/jboss-logging-3.1.3.GA.jar:lib/jets3t-0.9.4.jar:lib/log4j-api-2.3.jar:lib/aws-java-sdk-1.10.21.jar:lib/maven-model-builder-3.1.1.jar:lib/druid-aws-common-0.8.2.jar:lib/aws-java-sdk-elasticloadbalancing-1.10.21.jar:lib/extendedset-1.3.9.jar:lib/druid-indexing-service-0.8.2.jar:lib/druid-services-0.8.2.jar:lib/config-magic-0.9.jar:lib/jsr311-api-1.1.1.jar:lib/base64-2.3.8.jar:lib/aws-java-sdk-dynamodb-1.10.21.jar:lib/aws-java-sdk-ssm-1.10.21.jar:lib/druid-processing-0.8.2.jar:lib/aws-java-sdk-cloudhsm-1.10.21.jar:lib/jcl-over-slf4j-1.7.12.jar:lib/commons-logging-1.1.1.jar:lib/geoip2-0.4.0.jar:lib/aws-java-sdk-core-1.10.21.jar:lib/druid-console-0.0.2.jar:lib/aws-java-sdk-iam-1.10.21.jar:lib/jackson-jaxrs-json-provider-2.4.6.jar:lib/commons-io-2.0.1.jar:lib/hibernate-validator-5.1.3.Final.jar:lib/aws-java-sdk-devicefarm-1.10.21.jar:lib/javax.inject-1.jar:lib/javax.servlet-api-3.1.0.jar:lib/maven-repository-metadata-3.1.1.jar:lib/aws-java-sdk-logs-1.10.21.jar:lib/aws-java-sdk-ses-1.10.21.jar:lib/aws-java-sdk-cognitosync-1.10.21.jar:lib/aws-java-sdk-opsworks-1.10.21.jar:lib/log4j-core-2.3.jar:lib/maven-aether-provider-3.1.1.jar:lib/jetty-continuation-9.2.5.v20141112.jar:lib/aether-connector-file-0.9.0.M2.jar:lib/javax.el-api-3.0.0.jar:lib/airline-0.7.jar:lib/jetty-http-9.2.5.v20141112.jar:lib/aws-java-sdk-workspaces-1.10.21.jar:lib/jackson-core-2.4.6.jar:lib/aws-java-sdk-cloudtrail-1.10.21.jar:lib/guice-multibindings-4.0-beta.jar:lib/jersey-guice-1.19.jar:lib/aws-java-sdk-directory-1.10.21.jar:lib/aws-java-sdk-kinesis-1.10.21.jar:lib/irc-api-1.0-0014.jar:lib/annotations-2.0.3.jar:lib/RoaringBitmap-0.5.4.jar:lib/jackson-databind-2.4.6.jar:lib/jersey-servlet-1.19.jar:lib/emitter-0.3.2.jar:lib/aws-java-sdk-cloudwatch-1.10.21.jar:lib/aether-util-0.9.0.M2.jar:lib/druid-server-0.8.2.jar:lib/mapdb-1.0.8.jar:lib/aws-java-sdk-emr-1.10.21.jar:lib/okhttp-1.0.2.jar:lib/derbyclient-10.11.1.1.jar:lib/aws-java-sdk-efs-1.10.21.jar:lib/tesla-aether-0.0.5.jar:lib/jackson-datatype-guava-2.4.6.jar:lib/google-http-client-jackson2-1.15.0-rc.jar:lib/curator-x-discovery-2.8.0.jar:lib/jetty-servlet-9.2.5.v20141112.jar:lib/aws-java-sdk-kms-1.10.21.jar:lib/jetty-server-9.2.5.v20141112.jar:lib/aws-java-sdk-lambda-1.10.21.jar:lib/jackson-module-jaxb-annotations-2.4.6.jar:lib/aws-java-sdk-route53-1.10.21.jar:lib/log4j-jul-2.3.jar:lib/aether-spi-0.9.0.M2.jar:lib/aws-java-sdk-cloudsearch-1.10.21.jar:lib/aether-api-0.9.0.M2.jar:lib/commons-lang-2.6.jar:lib/icu4j-4.8.1.jar:lib/org.abego.treelayout.core-1.0.1.jar:lib/aws-java-sdk-ec2-1.10.21.jar:lib/httpclient-4.5.1.jar:lib/lz4-1.3.0.jar:lib/plexus-interpolation-1.19.jar:lib/guice-4.0-beta.jar:lib/activation-1.1.1.jar:lib/aws-java-sdk-sts-1.10.21.jar:lib/jackson-annotations-2.4.6.jar:lib/bcprov-jdk15on-1.52.jar:lib/commons-cli-1.2.jar:lib/jersey-server-1.19.jar:lib/aether-impl-0.9.0.M2.jar:lib/aws-java-sdk-ecs-1.10.21.jar:lib/aws-java-sdk-redshift-1.10.21.jar:lib/aws-java-sdk-swf-libraries-1.10.21.jar:lib/compress-lzf-1.0.3.jar:lib/aws-java-sdk-s3-1.10.21.jar:lib/httpcore-4.4.3.jar:lib/aws-java-sdk-codecommit-1.10.21.jar:lib/aws-java-sdk-codedeploy-1.10.21.jar:lib/jetty-io-9.2.5.v20141112.jar:lib/javax.el-3.0.0.jar:lib/curator-client-2.8.0.jar:lib/aws-java-sdk-config-1.10.21.jar:lib/aws-java-sdk-machinelearning-1.10.21.jar:lib/commons-codec-1.7.jar:lib/java-util-0.27.4.jar:lib/xpp3-1.1.4c.jar:lib/plexus-utils-3.0.15.jar:lib/commons-pool-1.6.jar:lib/aws-java-sdk-cloudfront-1.10.21.jar:lib/aws-java-sdk-autoscaling-1.10.21.jar:lib/protobuf-java-2.5.0.jar:lib/guice-servlet-4.0-beta.jar:lib/netty-3.10.4.Final.jar:lib/java-xmlbuilder-1.1.jar:lib/derby-10.11.1.1.jar:lib/google-http-client-1.15.0-rc.jar:lib/zookeeper-3.4.6.jar:lib/maven-model-3.1.1.jar:lib/curator-framework-2.8.0.jar:lib/aws-java-sdk-storagegateway-1.10.21.jar:lib/aws-java-sdk-cloudformation-1.10.21.jar:config/overlord:config/_common
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.version=3.13.0-71-generic
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.name=ritesh
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/ritesh
2015-12-17T05:44:59,970 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/ritesh/druid-0.8.2
2015-12-17T05:44:59,971 INFO [main] org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@2293e15d
2015-12-17T05:44:59,984 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.http.client.NettyHttpClient.start()] on object[com.metamx.http.client.NettyHttpClient@5f09668d].
2015-12-17T05:44:59,985 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.indexing.overlord.TaskMaster.start()] on object[io.druid.indexing.overlord.TaskMaster@2572b62a].
2015-12-17T05:44:59,986 INFO [main-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2015-12-17T05:44:59,988 INFO [main] org.eclipse.jetty.server.Server - jetty-9.2.5.v20141112
2015-12-17T05:44:59,993 INFO [main-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:2181, initiating session
2015-12-17T05:45:00,022 INFO [main-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x151ae77050f0007, negotiated timeout = 30000
2015-12-17T05:45:00,027 INFO [main-EventThread] org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
Dec 17, 2015 5:45:00 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider as a provider class
Dec 17, 2015 5:45:00 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering io.druid.server.StatusResource as a root resource class
Dec 17, 2015 5:45:00 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Dec 17, 2015 5:45:00 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider to GuiceManagedComponentProvider with the scope "Singleton"
Dec 17, 2015 5:45:00 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.indexing.overlord.http.OverlordResource to GuiceInstantiatedComponentProvider
Dec 17, 2015 5:45:00 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding io.druid.server.StatusResource to GuiceManagedComponentProvider with the scope "Undefined"
Dec 17, 2015 5:45:00 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
  WARNING: Parameter 1 of type io.druid.indexing.common.actions.TaskActionHolder<T> from public <T> javax.ws.rs.core.Response io.druid.indexing.overlord.http.OverlordResource.doAction(io.druid.indexing.common.actions.TaskActionHolder<T>) is not resolvable to a concrete type
2015-12-17T05:45:00,493 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@34863565{/,[jar:file:/home/ritesh/druid-0.8.2/lib/druid-server-0.8.2.jar!/static, jar:file:/home/ritesh/druid-0.8.2/lib/druid-indexing-service-0.8.2.jar!/indexer_static],AVAILABLE}
2015-12-17T05:45:00,499 INFO [main] org.eclipse.jetty.server.ServerConnector - Started ServerConnector@43fa4270{HTTP/1.1}{0.0.0.0:8090}
2015-12-17T05:45:00,500 INFO [main] org.eclipse.jetty.server.Server - Started @4479ms
2015-12-17T05:45:01,371 INFO [Curator-LeaderSelector-0] io.druid.indexing.overlord.TaskMaster - By the power of Grayskull, I have the power!
2015-12-17T05:45:01,372 INFO [Curator-LeaderSelector-0] io.druid.indexing.overlord.TaskLockbox - Synced 0 locks for 0 activeTasks from storage (0 locks ignored).
2015-12-17T05:45:01,382 INFO [Curator-LeaderSelector-0] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.indexing.overlord.RemoteTaskRunner.start()] on object[io.druid.indexing.overlord.RemoteTaskRunner@27e68659].
2015-12-17T05:45:01,504 INFO [Curator-LeaderSelector-0] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.indexing.overlord.TaskQueue.start()] on object[io.druid.indexing.overlord.TaskQueue@2a079c26].
2015-12-17T05:45:01,506 INFO [Curator-LeaderSelector-0] io.druid.indexing.overlord.TaskQueue - Synced 0 tasks from storage (0 tasks added, 0 tasks removed).
2015-12-17T05:45:01,508 INFO [TaskQueue-Manager] io.druid.indexing.overlord.TaskQueue - Beginning management in PT0S.
2015-12-17T05:45:01,513 INFO [Curator-LeaderSelector-0] io.druid.curator.discovery.CuratorServiceAnnouncer - Announcing service[DruidNode{serviceName='overlord', host='localhost', port=8090}]
2015-12-17T05:46:01,510 INFO [TaskQueue-StorageSync] io.druid.indexing.overlord.TaskQueue - Synced 0 tasks from storage (0 tasks added, 0 tasks removed).
2015-12-17T05:47:01,510 INFO [TaskQueue-StorageSync] io.druid.indexing.overlord.TaskQueue - Synced 0 tasks from storage (0 tasks added, 0 tasks removed).
2015-12-17T05:48:01,510 INFO [TaskQueue-StorageSync] io.druid.indexing.overlord.TaskQueue - Synced 0 tasks from storage (0 tasks added, 0 tasks removed).
2015-12-17T05:49:01,510 INFO [TaskQueue-StorageSync] io.druid.indexing.overlord.TaskQueue - Synced 0 tasks from storage (0 tasks added, 0 tasks removed).
2015-12-17T05:50:01,510 INFO [TaskQueue-StorageSync] io.druid.indexing.overlord.TaskQueue - Synced 0 tasks from storage (0 tasks added, 0 tasks removed).
2015-12-17T05:50:05,583 INFO [Curator-PathChildrenCache-0] io.druid.indexing.overlord.RemoteTaskRunner - Worker[localhost:8100] reportin' for duty!



 These are the steps i am following :
  1. starting zookeeper.
  2. Starting coordinator , broker, historical , overlord and middlemanager nodes.
  3. Running the code.
Then , when i do a select on the datasource it returns blank.

curl -X POST 'http://localhost:8082/druid/v2/?pretty' -H 'content-type: application/json' -d@/home/ritesh/druid-0.8.2/examples/tranq_test/select.json

content of /tranq_test/select.json

{
   "queryType": "select",
   "dataSource": "tranq_test",
   "dimensions":[],
   "metrics":[],
   "granularity": "all",
   "intervals": [
     "2014-01-01/2015-12-31"
   ],
   "pagingSpec":{"pagingIdentifiers": {}, "threshold":5}
 }


I am able to insert data into druid when i am using kafka(using realtime node). Also, i am able to query the data.I am attaching my properties file.Please have a look.

Thanks,


common.runtime.properties
###Broker runtime.properties

Ritesh Sinha

unread,
Dec 17, 2015, 3:59:03 AM12/17/15
to Druid User
Hi David,

On digging deep i found out this error .


2015-12-17T08:45:54,229 ERROR [task-runner-0] io.druid.curator.discovery.ServerDiscoverySelector - No server instance found
2015-12-17T08:45:54,229 WARN [task-runner-0] io.druid.indexing.common.actions.RemoteTaskActionClient - Exception submitting action for task[index_realtime_foo_2015-12-17T08:00:00.000Z_0_0]
java.io.IOException: Failed to locate service uri
at io.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:91) [druid-indexing-service-0.8.2.jar:0.8.2]
at io.druid.indexing.common.task.AbstractTask.getTaskLocks(AbstractTask.java:184) [druid-indexing-service-0.8.2.jar:0.8.2]
at io.druid.indexing.common.task.RealtimeIndexTask.run(RealtimeIndexTask.java:168) [druid-indexing-service-0.8.2.jar:0.8.2]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:221) [druid-indexing-service-0.8.2.jar:0.8.2]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:200) [druid-indexing-service-0.8.2.jar:0.8.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_91]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_91]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_91]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_91]
Caused by: com.metamx.common.ISE: Cannot find instance of indexer to talk to!
at io.druid.indexing.common.actions.RemoteTaskActionClient.getServiceInstance(RemoteTaskActionClient.java:165) ~[druid-indexing-service-0.8.2.jar:0.8.2]
at io.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:86) ~[druid-indexing-service-0.8.2.jar:0.8.2]
... 8 more
2015-12-17T08:45:54,238 INFO [task-runner-0] io.druid.indexing.common.actions.RemoteTaskActionClient - Will try again in [PT70.993S].
2015-12-17T08:47:05,231 ERROR [task-runner-0] io.druid.curator.discovery.ServerDiscoverySelector - No server instance found

David Lim

unread,
Dec 17, 2015, 12:28:50 PM12/17/15
to Druid User
Hey Ritesh,

Pretty sure your issue is with druid.discovery.curator.path. It's being set as /discovery on the overlord but is the default of /druid/discovery on the middle manager since it hasn't been set. I would recommend removing it from the overlord/runtime.properties and moving it into common.runtime.properties so it can be shared by all services.

Debashis Paul

unread,
May 29, 2018, 2:23:25 PM5/29/18
to Druid User
Hi David , 

Still no luck with the mentioned changes and when we start service below :

java `cat conf-quickstart/druid/historical/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/historical:lib/*" io.druid.cli.Main server historical

I am still getting this error :

 ERROR [main] io.druid.curator.discovery.ServerDiscoverySelector - No server instance found for [druid/coordinator]

any help will be appreciated. 

thanks 

Debashis Paul

unread,
May 29, 2018, 2:49:31 PM5/29/18
to Druid User
Finally found the problem with the sequence of process startup .

I did below sequence than the order mentioned in the document and everything loaded fine :

nohup java `cat conf-quickstart/druid/coordinator/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/coordinator:lib/*" io.druid.cli.Main server coordinator >> coordinator.log 2>&1 &

nohup java `cat conf-quickstart/druid/overlord/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/overlord:lib/*" io.druid.cli.Main server overlord >> overlord.log 2>&1 &

nohup java `cat conf-quickstart/druid/historical/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/historical:lib/*" io.druid.cli.Main server historical >> historical.log 2>&1 &

nohup java `cat conf-quickstart/druid/broker/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/broker:lib/*" io.druid.cli.Main server broker >> broker.log 2>&1 &

nohup java `cat conf-quickstart/druid/middleManager/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/middleManager:lib/*" io.druid.cli.Main server middleManager >> middlemanager.log 2>&1 &

nohup java `cat conf-quickstart/druid/realtime/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/realtime:lib/*" io.druid.cli.Main server realtime >> realtime.log 2>&1 &

thanks
Reply all
Reply to author
Forward
0 new messages