Hi all,
I try to get Titan running in a very simple environment with Cassandra as backend:
- Running Cassandra instance with authentication turned on
- On the same machine I run a jar that connects to Cassandra through Titan
- Cassandra has an empty keyspace called Titan
- user under which the application connects to Cassandra (through Titan) has all privileges on Cassandra keyspace "Titan" but is not allowed to create new keyspaces
When I start up my application the following tables are created in the keyspace Titan:
- edgeindex
- edgestore
- edgestore_lock_
- system_properties
- titan_ids
- vertexindex
- vertexindex_lock_
This looks to me like my application / Titan could successfully connect to Cassandra. However I never obtain the graph from Cassandra since I get the following Connection error (App calls TitanGraph graph = TitanFactory.open(config);):
Dec 01 2013 15:13:42 [INFO ] main ch.uzh.ifi.ddis.dm.twhc.input.TestDriver: Passed CommandLineArgs: [-db, titan.properties, -log, log4j.properties]
Dec 01 2013 15:13:42 [INFO ] main ch.uzh.ifi.ddis.dm.twhc.clusterhierarchy.persistence.BlueprintsSerializer: Setting up backend connection with the following properties:
{storage.password=XXXXXXXX, storage.backend=cassandra, storage.username=uzh, storage.hostname=127.0.0.1}
Dec 01 2013 15:13:42 [INFO ] main ch.uzh.ifi.ddis.dm.twhc.graphdb.GraphDbConnectorImpl: Titan is called for graph with the following configuration:
storage.password=XXXXXXXX
storage.backend=cassandra
storage.username=uzh
storage.hostname=127.0.0.1
Dec 01 2013 15:13:42 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:42 [DEBUG] main com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager: About to instantiate class public com.netflix.astyanax.retry.BoundedExponentialBackoff(int,int,int) with 3 arguments
Dec 01 2013 15:13:42 [DEBUG] main com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager: Instantiated RetryPolicy object com.netflix.astyanax.retry.BoundedExponentialBackoff@7687ac8f[maxSleepTimeMs=25000,random=java.util.Random@38e72741,baseSleepTimeMs=100,maxAttempts=8,attempts=0] from config string "com.netflix.astyanax.retry.BoundedExponentialBackoff,100,25000,8"
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager: Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,name=ClusterTitanConnectionPool,ServiceType=connectionpool
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: AddHost: 127.0.0.1
Dec 01 2013 15:13:42 [DEBUG] main com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager: Found keyspace titan
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager: Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,name=KeyspaceTitanConnectionPool,ServiceType=connectionpool
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: AddHost: 127.0.0.1
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: AddHost: 130.60.156.101
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: AddHost: 130.60.156.102
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: RemoveHost: 127.0.0.1
Dec 01 2013 15:13:43 [INFO ] main com.thinkaurelius.titan.diskstorage.Backend: Initiated backend operations thread pool of size 2
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.locking.LocalLockMediators: Local lock mediator instantiated for namespace AstyanaxStoreManagertitan
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:13:53 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:03 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:13 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:13 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:23 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:23 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:24 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:25 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:26 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:27 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:28 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:29 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:30 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:31 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:32 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
….
….
Dec 01 2013 15:15:12 [DEBUG] RetryService : 130.60.156.102(130.60.156.102):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:15:12 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
com.thinkaurelius.titan.core.TitanException: Could not successfully complete backend operation due to repeated temporary exceptions after 60000 ms
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:49)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration.getConfigurationProperty(KCVSConfiguration.java:64)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:268)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1109)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at ch.uzh.ifi.ddis.dm.twhc.graphdb.GraphDbConnectorImpl.connect(GraphDbConnectorImpl.java:43)
at ch.uzh.ifi.ddis.dm.twhc.clusterhierarchy.persistence.BlueprintsSerializer.<init>(BlueprintsSerializer.java:94)
at ch.uzh.ifi.ddis.dm.twhc.input.TestDriver.main(TestDriver.java:166)
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryStorageException: Temporary failure in storage backend
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getNamesSlice(AstyanaxOrderedKeyColumnValueStore.java:138)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getSlice(AstyanaxOrderedKeyColumnValueStore.java:88)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration$1.call(KCVSConfiguration.java:70)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration$1.call(KCVSConfiguration.java:64)
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:30)
... 8 more
Caused by: com.netflix.astyanax.connectionpool.exceptions.NoAvailableHostsException: NoAvailableHostsException: [host=None(0.0.0.0):0, latency=0(0), attempts=0]No hosts to borrow from
at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.<init>(RoundRobinExecuteWithFailover.java:30)
at com.netflix.astyanax.connectionpool.impl.TokenAwareConnectionPoolImpl.newExecuteWithFailover(TokenAwareConnectionPoolImpl.java:83)
at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:256)
at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4.execute(ThriftColumnFamilyQueryImpl.java:519)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getNamesSlice(AstyanaxOrderedKeyColumnValueStore.java:136)
... 12 more
Dec 01 2013 15:15:13 [ERROR] main org.slf4j.helpers.MarkerIgnoringBase: Could not successfully complete backend operation due to repeated temporary exceptions after 60000 ms
com.thinkaurelius.titan.core.TitanException: Could not successfully complete backend operation due to repeated temporary exceptions after 60000 ms
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:49)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration.getConfigurationProperty(KCVSConfiguration.java:64)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:268)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1109)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at ch.uzh.ifi.ddis.dm.twhc.graphdb.GraphDbConnectorImpl.connect(GraphDbConnectorImpl.java:43)
at ch.uzh.ifi.ddis.dm.twhc.clusterhierarchy.persistence.BlueprintsSerializer.<init>(BlueprintsSerializer.java:94)
at ch.uzh.ifi.ddis.dm.twhc.input.TestDriver.main(TestDriver.java:166)
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryStorageException: Temporary failure in storage backend
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getNamesSlice(AstyanaxOrderedKeyColumnValueStore.java:138)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getSlice(AstyanaxOrderedKeyColumnValueStore.java:88)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration$1.call(KCVSConfiguration.java:70)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration$1.call(KCVSConfiguration.java:64)
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:30)
... 8 more
Caused by: com.netflix.astyanax.connectionpool.exceptions.NoAvailableHostsException: NoAvailableHostsException: [host=None(0.0.0.0):0, latency=0(0), attempts=0]No hosts to borrow from
at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.<init>(RoundRobinExecuteWithFailover.java:30)
at com.netflix.astyanax.connectionpool.impl.TokenAwareConnectionPoolImpl.newExecuteWithFailover(TokenAwareConnectionPoolImpl.java:83)
at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:256)
at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4.execute(ThriftColumnFamilyQueryImpl.java:519)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getNamesSlice(AstyanaxOrderedKeyColumnValueStore.java:136)
... 12 more
Application terminated prematurely!
On my local set up the application runs fine hence I assume it is a configuration problem (of Cassandra?).
Do you have an idea what could cause this connection problem?
Thanks for your hints and happy Sunday,
Best Faber