Error connecting to localhost:1,527 with message Connection refused when starting Indexing service

1,372 views
Skip to first unread message

Mahesh Abnave

unread,
Jun 28, 2015, 3:29:53 PM6/28/15
to druid...@googlegroups.com
Hi I am trying out batch ingestion from tut here.

Environment
  • I am on Fedora 20 x64 running on VMWare.
  • Java details
    java version "1.7.0_45"
    OpenJDK Runtime Environment (fedora-2.4.3.0.fc20-x86_64 u45-b15)
    OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
  • Zookeeper 3.4.6
  • druid 0.7.3
$DRUID_HOME\Config\_common\common.runtime.properties

# Extensions (no deep storage model is listed - using local fs for deep storage - not recommended for production)
druid.extensions.coordinates=["io.druid.extensions:druid-examples","io.druid.extensions:druid-kafka-eight","io.druid.extensions:mysql-metadata-storage"]

# Zookeeper
druid.zk.service.host=localhost

# Metadata Storage (mysql)
druid.metadata.storage.type=mysql
druid.metadata.storage.connector.connectURI=jdbc\:mysql\://localhost\:3306/druid
druid.metadata.storage.connector.user=druid
druid.metadata.storage.connector.password=diurd

# Deep storage (local filesystem for examples - don't use this in production)
druid.storage.type=local
druid.storage.storageDirectory=/tmp/druid/localStorage

# Query Cache (we use a simple 10mb heap-based local cache on the broker)
druid.cache.type=local
druid.cache.sizeInBytes=10000000

# Indexing service discovery
druid.selectors.indexing.serviceName=overlord

# Monitoring (disabled for examples, if you enable SysMonitor, make sure to include sigar jar in your cp)
# druid.monitoring.monitors=["com.metamx.metrics.SysMonitor","com.metamx.metrics.JvmMonitor"]

# Metrics logging (disabled for examples - change this to logging or http in production)
druid.emitter=noop

 Steps I followed
  1. MySQL Setup
    • Installed mysql
    • Then did following:
      sudo service mysqld start
      mysql -u root
      mysql> GRANT ALL ON druid.* TO 'druid'@'localhost' IDENTIFIED BY 'diurd';
      mysql> CREATE DATABASE druid DEFAULT CHARACTER SET utf8;
      All went fine. Checked in MySQL, Workbench druid user and database is created.

  2. Started coordinator node
    java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath config/_common:config/coordinator:lib/* io.druid.cli.Main server coordinator

  3. Started historical node
    java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath config/_common:config/historical:lib/* io.druid.cli.Main server historical

  4. Started indexing service
    java -Xmx2g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/overlord io.druid.cli.Main server overlord 

    Ended up with following error
1    $ java -Xmx2g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/overlord io.druid.cli.Main server overlord 
2    2015-06-25T20:47:08,615 INFO [main] io.druid.guice.PropertiesModule - Loading properties from runtime.properties
3    Jun 25, 2015 8:47:08 PM org.hibernate.validator.internal.util.Version <clinit>
4    INFO: HV000001: Hibernate Validator 5.1.3.Final
5    2015-06-25T20:47:09,923 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[], defaultVersion='0.7.3', localRepository='/home/mahesha999/.m2/repository', remoteRepositories=[https://repo1.maven.org/maven2/, https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local]}]
6    2015-06-25T20:47:12,598 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.metadata.MetadataStorageConnectorConfig] from props[druid.metadata.storage.connector.] as [DbConnectorConfig{createTables=true, connectURI='jdbc:derby://localhost:1527/druid;create=true', user='null', passwordProvider=null}]
7    2015-06-25T20:47:12,633 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.metadata.MetadataStorageTablesConfig] from props[druid.metadata.storage.tables.] as [io.druid.metadata.MetadataStorageTablesConfig@7c92fffb]
8    2015-06-25T20:47:12,894 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.common.config.ConfigManagerConfig] from props[druid.manager.config.] as [io.druid.common.config.ConfigManagerConfig@2a1a3ee9]
9    2015-06-25T20:47:12,983 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.DruidNode] from props[druid.] as [DruidNode{serviceName='overlord', host='localhost', port=8090}]
10    2015-06-25T20:47:12,990 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.audit.SQLAuditManagerConfig] from props[druid.audit.manager.] as [io.druid.server.audit.SQLAuditManagerConfig@5e5efbee]
11    2015-06-25T20:47:13,012 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@3ae3a5d0]
12    2015-06-25T20:47:13,031 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[]}]
13    2015-06-25T20:47:13,070 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ServerConfig] from props[druid.server.http.] as [ServerConfig{numThreads=40, maxIdleTime=PT5M}]
14    2015-06-25T20:47:13,096 INFO [main] org.eclipse.jetty.util.log - Logging initialized @9648ms
15    2015-06-25T20:47:13,346 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.config.TaskQueueConfig] from props[druid.indexer.queue.] as [io.druid.indexing.overlord.config.TaskQueueConfig@6fa61bae]
16    2015-06-25T20:47:13,357 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.TaskStorageConfig] from props[druid.indexer.storage.] as [io.druid.indexing.common.config.TaskStorageConfig@550b7a01]
17    2015-06-25T20:47:13,395 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ZkPathsConfig] from props[druid.zk.paths.] as [io.druid.server.initialization.ZkPathsConfig@58d3f4be]
18    2015-06-25T20:47:13,414 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.IndexerZkConfig] from props[druid.zk.paths.indexer.] as [io.druid.server.initialization.IndexerZkConfig@d5bd5202]
19    2015-06-25T20:47:13,431 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.config.ForkingTaskRunnerConfig] from props[druid.indexer.runner.] as [io.druid.indexing.overlord.config.ForkingTaskRunnerConfig@3fb85e58]
20    2015-06-25T20:47:13,448 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.TaskConfig] from props[druid.indexer.task.] as [io.druid.indexing.common.config.TaskConfig@5694a0ba]
21    2015-06-25T20:47:13,457 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.worker.config.WorkerConfig] from props[druid.worker.] as [io.druid.indexing.worker.config.WorkerConfig@50e3fa1d]
22    2015-06-25T20:47:13,472 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.FileTaskLogsConfig] from props[druid.indexer.logs.] as [io.druid.indexing.common.config.FileTaskLogsConfig@625f8cc7]
23    2015-06-25T20:47:13,491 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.autoscaling.SimpleResourceManagementConfig] from props[druid.indexer.autoscale.] as [io.druid.indexing.overlord.autoscaling.SimpleResourceManagementConfig@4708fb1c]
24    2015-06-25T20:47:13,492 INFO [main] io.druid.common.config.ConfigManager - Creating watch for key[worker.config]
25    2015-06-25T20:47:13,562 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.autoscaling.ResourceManagementSchedulerConfig] from props[druid.indexer.autoscale.] as [io.druid.indexing.overlord.autoscaling.ResourceManagementSchedulerConfig@bf6d6ca]
26    2015-06-25T20:47:13,570 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.curator.CuratorConfig] from props[druid.zk.service.] as [io.druid.curator.CuratorConfig@691618bc]
27    2015-06-25T20:47:13,577 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29
28    2015-06-25T20:47:13,677 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.CuratorDiscoveryConfig] from props[druid.discovery.curator.] as [io.druid.server.initialization.CuratorDiscoveryConfig@533f6c57]
29    2015-06-25T20:47:14,187 WARN [main] io.druid.metadata.SQLMetadataConnector - Exception creating table
30    org.skife.jdbi.v2.exceptions.UnableToObtainConnectionException: java.sql.SQLException: Cannot create PoolableConnectionFactory (java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.)
31        at org.skife.jdbi.v2.DBI.open(DBI.java:210) ~[jdbi-2.32.jar:?]
32        at org.skife.jdbi.v2.DBI.withHandle(DBI.java:257) ~[jdbi-2.32.jar:?]
33        at io.druid.metadata.SQLMetadataConnector.createTable(SQLMetadataConnector.java:88) [druid-server-0.7.3.jar:0.7.3]
34        at io.druid.metadata.SQLMetadataConnector.createConfigTable(SQLMetadataConnector.java:164) [druid-server-0.7.3.jar:0.7.3]
35        at io.druid.metadata.SQLMetadataConnector.createConfigTable(SQLMetadataConnector.java:311) [druid-server-0.7.3.jar:0.7.3]
36        at io.druid.guice.JacksonConfigManagerModule$1.start(JacksonConfigManagerModule.java:56) [druid-common-0.7.3.jar:0.7.3]
37        at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:250) [java-util-0.27.0.jar:?]
38        at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:136) [druid-api-0.3.8.jar:0.7.3]
39        at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:71) [druid-services-0.7.3.jar:0.7.3]
40        at io.druid.cli.ServerRunnable.run(ServerRunnable.java:38) [druid-services-0.7.3.jar:0.7.3]
41        at io.druid.cli.Main.main(Main.java:88) [druid-services-0.7.3.jar:0.7.3]
42    Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory (java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.)
43        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2152) ~[commons-dbcp2-2.0.1.jar:2.0.1]
44        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
45        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
46        at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.32.jar:?]
47        at org.skife.jdbi.v2.DBI.open(DBI.java:192) ~[jdbi-2.32.jar:?]
48        ... 10 more
49    Caused by: java.sql.SQLNonTransientConnectionException: java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.
50        at org.apache.derby.client.am.SQLExceptionFactory.getSQLException(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
51        at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
52        at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
53        at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39) ~[commons-dbcp2-2.0.1.jar:2.0.1]
54        at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) ~[commons-dbcp2-2.0.1.jar:2.0.1]
55        at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2162) ~[commons-dbcp2-2.0.1.jar:2.0.1]
56        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2148) ~[commons-dbcp2-2.0.1.jar:2.0.1]
57        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
58        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
59        at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.32.jar:?]
60        at org.skife.jdbi.v2.DBI.open(DBI.java:192) ~[jdbi-2.32.jar:?]
61        ... 10 more
62    Caused by: org.apache.derby.client.am.DisconnectException: java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.
63        at org.apache.derby.client.net.NetAgent.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
64        at org.apache.derby.client.net.NetConnection.newAgent_(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
65        at org.apache.derby.client.am.ClientConnection.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
66        at org.apache.derby.client.net.NetConnection.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
67        at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
68        at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
69        at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39) ~[commons-dbcp2-2.0.1.jar:2.0.1]
70        at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) ~[commons-dbcp2-2.0.1.jar:2.0.1]
71        at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2162) ~[commons-dbcp2-2.0.1.jar:2.0.1]
72        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2148) ~[commons-dbcp2-2.0.1.jar:2.0.1]
73        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
74        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
75        at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.32.jar:?]
76        at org.skife.jdbi.v2.DBI.open(DBI.java:192) ~[jdbi-2.32.jar:?]
77        ... 10 more
78    Caused by: java.net.ConnectException: Connection refused
79        at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.7.0_45]
80        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) ~[?:1.7.0_45]
81        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) ~[?:1.7.0_45]
82        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) ~[?:1.7.0_45]
83        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.7.0_45]
84        at java.net.Socket.connect(Socket.java:579) ~[?:1.7.0_45]
85        at java.net.Socket.connect(Socket.java:528) ~[?:1.7.0_45]
86        at java.net.Socket.<init>(Socket.java:425) ~[?:1.7.0_45]
87        at java.net.Socket.<init>(Socket.java:208) ~[?:1.7.0_45]
88        at javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:271) ~[?:1.7.0_45]
89        at org.apache.derby.client.net.OpenSocketAction.run(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
90        at org.apache.derby.client.net.OpenSocketAction.run(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
91        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_45]
92        at org.apache.derby.client.net.NetAgent.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
93        at org.apache.derby.client.net.NetConnection.newAgent_(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
94        at org.apache.derby.client.am.ClientConnection.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
95        at org.apache.derby.client.net.NetConnection.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
96        at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
97        at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
98        at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39) ~[commons-dbcp2-2.0.1.jar:2.0.1]
99        at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) ~[commons-dbcp2-2.0.1.jar:2.0.1]
100        at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2162) ~[commons-dbcp2-2.0.1.jar:2.0.1]
101        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2148) ~[commons-dbcp2-2.0.1.jar:2.0.1]
102        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
103        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
104        at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.32.jar:?]
105        at org.skife.jdbi.v2.DBI.open(DBI.java:192) ~[jdbi-2.32.jar:?]
106        ... 10 more
107    2015-06-25T20:47:14,207 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.common.config.ConfigManager.start()] on object[io.druid.common.config.ConfigManager@4bdc7b2f].
108    2015-06-25T20:47:14,210 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.emitter.service.ServiceEmitter.start()] on object[com.metamx.emitter.service.ServiceEmitter@6b2284a4].
109    2015-06-25T20:47:14,212 WARN [config-manager-0] io.druid.common.config.ConfigManager - Exception when checking property[worker.config]
110    org.skife.jdbi.v2.exceptions.UnableToObtainConnectionException: java.sql.SQLException: Cannot create PoolableConnectionFactory (java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.)
111        at org.skife.jdbi.v2.DBI.open(DBI.java:210) ~[jdbi-2.32.jar:?]
112        at org.skife.jdbi.v2.DBI.withHandle(DBI.java:257) ~[jdbi-2.32.jar:?]
113        at io.druid.metadata.SQLMetadataConnector.lookup(SQLMetadataConnector.java:337) ~[druid-server-0.7.3.jar:0.7.3]
114        at io.druid.common.config.ConfigManager.poll(ConfigManager.java:106) [druid-common-0.7.3.jar:0.7.3]
115        at io.druid.common.config.ConfigManager.access$600(ConfigManager.java:42) [druid-common-0.7.3.jar:0.7.3]
116        at io.druid.common.config.ConfigManager$PollingCallable.call(ConfigManager.java:248) [druid-common-0.7.3.jar:0.7.3]
117        at io.druid.common.config.ConfigManager$PollingCallable.call(ConfigManager.java:232) [druid-common-0.7.3.jar:0.7.3]
118        at com.metamx.common.concurrent.ScheduledExecutors$2.run(ScheduledExecutors.java:99) [java-util-0.27.0.jar:?]
119        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [?:1.7.0_45]
120        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_45]
121        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) [?:1.7.0_45]
122        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) [?:1.7.0_45]
123        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_45]
124        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_45]
125        at java.lang.Thread.run(Thread.java:744) [?:1.7.0_45]
126    Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory (java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.)
127        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2152) ~[commons-dbcp2-2.0.1.jar:2.0.1]
128        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
129        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
130        at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.32.jar:?]
131        at org.skife.jdbi.v2.DBI.open(DBI.java:192) ~[jdbi-2.32.jar:?]
132        ... 14 more
133    Caused by: java.sql.SQLNonTransientConnectionException: java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.
134        at org.apache.derby.client.am.SQLExceptionFactory.getSQLException(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
135        at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
136        at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
137        at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39) ~[commons-dbcp2-2.0.1.jar:2.0.1]
138        at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) ~[commons-dbcp2-2.0.1.jar:2.0.1]
139        at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2162) ~[commons-dbcp2-2.0.1.jar:2.0.1]
140        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2148) ~[commons-dbcp2-2.0.1.jar:2.0.1]
141        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
142        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
143        at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.32.jar:?]
144        at org.skife.jdbi.v2.DBI.open(DBI.java:192) ~[jdbi-2.32.jar:?]
145        ... 14 more
146    Caused by: org.apache.derby.client.am.DisconnectException: java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.
147        at org.apache.derby.client.net.NetAgent.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
148        at org.apache.derby.client.net.NetConnection.newAgent_(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
149        at org.apache.derby.client.am.ClientConnection.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
150        at org.apache.derby.client.net.NetConnection.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
151        at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
152        at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
153        at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39) ~[commons-dbcp2-2.0.1.jar:2.0.1]
154        at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) ~[commons-dbcp2-2.0.1.jar:2.0.1]
155        at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2162) ~[commons-dbcp2-2.0.1.jar:2.0.1]
156        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2148) ~[commons-dbcp2-2.0.1.jar:2.0.1]
157        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
158        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
159        at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.32.jar:?]
160        at org.skife.jdbi.v2.DBI.open(DBI.java:192) ~[jdbi-2.32.jar:?]
161        ... 14 more
162    Caused by: java.net.ConnectException: Connection refused
163        at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.7.0_45]
164        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) ~[?:1.7.0_45]
165        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) ~[?:1.7.0_45]
166        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) ~[?:1.7.0_45]
167        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.7.0_45]
168        at java.net.Socket.connect(Socket.java:579) ~[?:1.7.0_45]
169        at java.net.Socket.connect(Socket.java:528) ~[?:1.7.0_45]
170        at java.net.Socket.<init>(Socket.java:425) ~[?:1.7.0_45]
171        at java.net.Socket.<init>(Socket.java:208) ~[?:1.7.0_45]
172        at javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:271) ~[?:1.7.0_45]
173        at org.apache.derby.client.net.OpenSocketAction.run(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
174        at org.apache.derby.client.net.OpenSocketAction.run(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
175        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_45]
176        at org.apache.derby.client.net.NetAgent.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
177        at org.apache.derby.client.net.NetConnection.newAgent_(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
178        at org.apache.derby.client.am.ClientConnection.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
179        at org.apache.derby.client.net.NetConnection.<init>(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
180        at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
181        at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) ~[derbyclient-10.11.1.1.jar:?]
182        at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39) ~[commons-dbcp2-2.0.1.jar:2.0.1]
183        at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) ~[commons-dbcp2-2.0.1.jar:2.0.1]
184        at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2162) ~[commons-dbcp2-2.0.1.jar:2.0.1]
185        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2148) ~[commons-dbcp2-2.0.1.jar:2.0.1]
186        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1903) ~[commons-dbcp2-2.0.1.jar:2.0.1]
187        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1413) ~[commons-dbcp2-2.0.1.jar:2.0.1]
188        at org.skife.jdbi.v2.DataSourceConnectionFactory.openConnection(DataSourceConnectionFactory.java:36) ~[jdbi-2.32.jar:?]
189        at org.skife.jdbi.v2.DBI.open(DBI.java:192) ~[jdbi-2.32.jar:?]
190        ... 14 more
191    

When I run:
netstat -a | grep 1527
It gives nothing, that means no process running on 1527. Am I missing to explicitly run some process?

druid%20exception%20logs[1].txt

Himanshu

unread,
Jun 28, 2015, 4:41:25 PM6/28/15
to druid...@googlegroups.com
with extensions, I guess you would need the version as well.

for druid.extensions.coordinates, try putting version alongside extension names for example "io.druid.extensions:mysql-metadata-storage:0.7.3"

also, for this experiment where everything is running on one machine, you don't really need to run mysql and druid will use derby by default which should be good enough. that can be done by removing mysql-metadata-storage extension from druid.extensions.coordinates and removing following properties


# Metadata Storage (mysql)
druid.metadata.storage.type=mysql
druid.metadata.storage.connector.connectURI=jdbc\:mysql\://localhost\:3306/druid
druid.metadata.storage.connector.user=druid
druid.metadata.storage.connector.password=diurd

-- Himanshu

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/e3320fcb-d64f-4916-a1f4-5c8f3630b8b2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Mahesh Abnave

unread,
Jun 28, 2015, 5:06:14 PM6/28/15
to druid...@googlegroups.com
Ok

removed mysql-metadata-storage extension from druid.extensions.coordinates and 
commented following properties

# Metadata Storage (mysql)
druid.metadata.storage.type=mysql
druid.metadata.storage.connector.connectURI=jdbc\:mysql\://localhost\:3306/druid
druid.metadata.storage.connector.user=druid
druid.metadata.storage.connector.password=diurd

Still getting same error. I guess, point is no process is running on port 1527.

Himanshu

unread,
Jun 28, 2015, 6:05:08 PM6/28/15
to druid...@googlegroups.com
ok, on another closer look I see that overlord does not have "config/_common" in the classpath, can you add that and try?

if you are not using mysql then make sure you start coordinator before overlord.

-- Himanshu

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/16f1349d-bed4-4594-aaaf-05e52c8ecdce%40googlegroups.com.

Himanshu

unread,
Jun 28, 2015, 6:11:26 PM6/28/15
to druid...@googlegroups.com
also, to correct myself regarding version in extension name, version is not essential and example config is ok.

i think your problem is just that config/_common is not in the overlord classpath.

-- himanshu

Mahesh Abnave

unread,
Jun 29, 2015, 10:00:01 AM6/29/15
to druid...@googlegroups.com
ok I am now on different installation of fedora on different machine and I have removed the kafka and examples extensions from the default common.runtime.properties. This is how it looks:


# Extensions (no deep storage model is listed - using local fs for deep storage - not recommended for production)
#druid.extensions.coordinates=["io.druid.extensions:druid-examples","io.druid.extensions:druid-kafka-eight","io.druid.extensions:mysql-metadata-storage"]
#druid.extensions.coordinates=["io.druid.extensions:druid-examples","io.druid.extensions:druid-kafka-eight"]


# Zookeeper
druid.zk.service.host=localhost

# Metadata Storage (mysql)
#druid.metadata.storage.type=mysql
#druid.metadata.storage.connector.connectURI=jdbc\:mysql\://localhost\:3306/druid
#druid.metadata.storage.connector.user=druid
#druid.metadata.storage.connector.password=diurd


# Deep storage (local filesystem for examples - don't use this in production)
druid.storage.type=local
druid.storage.storageDirectory=/tmp/druid/localStorage

# Query Cache (we use a simple 10mb heap-based local cache on the broker)
druid.cache.type=local
druid.cache.sizeInBytes=10000000

# Indexing service discovery
druid.selectors.indexing.serviceName=overlord

# Monitoring (disabled for examples, if you enable SysMonitor, make sure to include sigar jar in your cp)
# druid.monitoring.monitors=["com.metamx.metrics.SysMonitor","com.metamx.metrics.JvmMonitor"]

# Metrics logging (disabled for examples - change this to logging or http in production)
druid.emitter=noop

I started zookeeper. And surprisingly, now running coordinator first and it still gives me same error, now in coordinator not in indexer.

I will try adding config/_common on that old machine again once I reach home, but what I feel is that if it is trying to connect 1527
and there is no process listening on that port, then it will still throw exception.

Between it is confusing that it gives same error while starting coordinator. Is derby failing to start? Or something like that?

logs of starting coordinator attached.
druid 1527 err.txt

Eric Tschetter

unread,
Jun 29, 2015, 11:52:31 AM6/29/15
to druid...@googlegroups.com
In your initial description you said that you setup MySQL and then are
getting an error that nothing is listening on 1527. 1527 is the
default Derby port, MySQL should be on 3306 by default. In the logs,
it shows that it is trying to connect to derby as well:

2015-06-25T20:47:12,598 INFO [main] io.druid.guice.JsonConfigurator -
Loaded class[class io.druid.metadata.MetadataStorageConnectorConfig]
from props[druid.metadata.storage.connector.] as
[DbConnectorConfig{createTables=true,
connectURI='jdbc:derby://localhost:1527/druid;create=true',
user='null', passwordProvider=null}]

As Himanshu noted, your command-line for the indexer didn't have the
"_common" config in the classpath, so it's likely that in this case
the problem was that your indexer wasn't getting the config to use
MySQL.

Now that you've commented out the config to use MySQL, it *should* be
defaulting to Derby, but, tbh I've never used Derby with running this
and am not sure how well it's been tested so I cannot say anything to
its readiness.

I *think* that if you go back to when you were running MySQL/MariaDB,
uncomment the lines pointing to mysql and make sure to include that
file on the classpath, it will work.

--Eric
> --
> You received this message because you are subscribed to the Google Groups
> "Druid User" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-user+...@googlegroups.com.
> To post to this group, send email to druid...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/druid-user/d62acee2-8314-41c8-aac8-5f07079366f4%40googlegroups.com.

Himanshu

unread,
Jun 29, 2015, 12:37:42 PM6/29/15
to druid...@googlegroups.com
I tried same and it is working fine for me. Coordinator starts the derby process which starts listening on 1527 . I see in the logs that it was started..

---
2015-06-29T11:41:05,820 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.metadata.storage.derby.DerbyMetadataStorage.start()] on object[io.druid.metadata.storage.derby.DerbyMetadataStorage@65f2f9b0].
2015-06-29T11:41:05,820 INFO [main] io.druid.metadata.storage.derby.DerbyMetadataStorage - Starting Derby Metadata Storage
----

What version of druid are you using?

-- Himanshu

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/CAB8U%2Bh0kLvtFYseraAf0E3kQ9GeXABNOKS66K9xiWaDHjX9r9g%40mail.gmail.com.

Mahesh Abnave

unread,
Jun 30, 2015, 5:08:11 PM6/30/15
to druid...@googlegroups.com
Hi Eric and Himanshu,

Yes it started working very well as expected once I added config/_common to classpath.
Thank you.

Mahesh Abnave

unread,
Sep 21, 2015, 8:36:36 AM9/21/15
to Druid User
Ok coming back to this thread again, I dont know if I was following wrong tut, but I tried the command under 
title "Batch Ingestion Using the Indexing Service" on page:
    http://druid.io/docs/0.7.3/Batch-ingestion.html

Seems that it misses config/_common in classpath

Fangjin Yang

unread,
Sep 22, 2015, 12:03:49 PM9/22/15
to Druid User
Oops, that is a bug in the documentation. Do you mind submitting a PR with the fix?
Reply all
Reply to author
Forward
0 new messages