Connection to Cassandra dropped after init of Titan Keyspace

439 views
Skip to first unread message

Faber Chri

unread,
Dec 1, 2013, 9:47:38 AM12/1/13
to aureliu...@googlegroups.com
Hi all,

I try to get Titan running in a very simple environment with Cassandra as backend:
- Running Cassandra instance with authentication turned on
- On the same machine I run a jar that connects to Cassandra through Titan
- Cassandra has an empty keyspace called Titan
- user under which the application connects to Cassandra (through Titan) has all privileges on Cassandra keyspace "Titan" but is not allowed to create new keyspaces

When I start up my application the following tables are created in the keyspace Titan:
  • edgeindex
  • edgestore
  • edgestore_lock_
  • system_properties
  • titan_ids
  • vertexindex
  • vertexindex_lock_
This looks to me like my application / Titan could successfully connect to Cassandra. However I never obtain the graph from Cassandra since I get the following Connection error (App calls TitanGraph graph = TitanFactory.open(config);):

Dec 01 2013 15:13:42 [INFO ] main ch.uzh.ifi.ddis.dm.twhc.input.TestDriver: Passed CommandLineArgs: [-db, titan.properties, -log, log4j.properties]
Dec 01 2013 15:13:42 [INFO ] main ch.uzh.ifi.ddis.dm.twhc.clusterhierarchy.persistence.BlueprintsSerializer: Setting up backend connection with the following properties:
{storage.password=XXXXXXXX, storage.backend=cassandra, storage.username=uzh, storage.hostname=127.0.0.1}
Dec 01 2013 15:13:42 [INFO ] main ch.uzh.ifi.ddis.dm.twhc.graphdb.GraphDbConnectorImpl: Titan is called for graph with the following configuration:
storage.password=XXXXXXXX
storage.backend=cassandra
storage.username=uzh
storage.hostname=127.0.0.1
Dec 01 2013 15:13:42 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:42 [DEBUG] main com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager: About to instantiate class public com.netflix.astyanax.retry.BoundedExponentialBackoff(int,int,int) with 3 arguments
Dec 01 2013 15:13:42 [DEBUG] main com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager: Instantiated RetryPolicy object com.netflix.astyanax.retry.BoundedExponentialBackoff@7687ac8f[maxSleepTimeMs=25000,random=java.util.Random@38e72741,baseSleepTimeMs=100,maxAttempts=8,attempts=0] from config string "com.netflix.astyanax.retry.BoundedExponentialBackoff,100,25000,8"
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager: Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,name=ClusterTitanConnectionPool,ServiceType=connectionpool
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: AddHost: 127.0.0.1
Dec 01 2013 15:13:42 [DEBUG] main com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager: Found keyspace titan
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager: Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,name=KeyspaceTitanConnectionPool,ServiceType=connectionpool
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: AddHost: 127.0.0.1
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: AddHost: 130.60.156.101
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: AddHost: 130.60.156.102
Dec 01 2013 15:13:42 [INFO ] main com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor: RemoveHost: 127.0.0.1
Dec 01 2013 15:13:43 [INFO ] main com.thinkaurelius.titan.diskstorage.Backend: Initiated backend operations thread pool of size 2
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.locking.LocalLockMediators: Local lock mediator instantiated for namespace AstyanaxStoreManagertitan
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] main com.thinkaurelius.titan.diskstorage.common.DistributedStoreManager: Set rid: 0x7f00010131403736313039407669746f
Dec 01 2013 15:13:43 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:13:53 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:03 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:13 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:13 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:23 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:23 [DEBUG] pool-4-thread-1 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:24 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:25 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:26 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:27 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:28 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:29 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:30 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:31 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:14:32 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
….
….
Dec 01 2013 15:15:12 [DEBUG] RetryService : 130.60.156.102(130.60.156.102):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
Dec 01 2013 15:15:12 [DEBUG] RetryService : 130.60.156.101(130.60.156.101):9160 com.netflix.astyanax.thrift.ThriftConverter: java.net.ConnectException: Connection refused
com.thinkaurelius.titan.core.TitanException: Could not successfully complete backend operation due to repeated temporary exceptions after 60000 ms
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:49)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration.getConfigurationProperty(KCVSConfiguration.java:64)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:268)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1109)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at ch.uzh.ifi.ddis.dm.twhc.graphdb.GraphDbConnectorImpl.connect(GraphDbConnectorImpl.java:43)
at ch.uzh.ifi.ddis.dm.twhc.clusterhierarchy.persistence.BlueprintsSerializer.<init>(BlueprintsSerializer.java:94)
at ch.uzh.ifi.ddis.dm.twhc.input.TestDriver.main(TestDriver.java:166)
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryStorageException: Temporary failure in storage backend
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getNamesSlice(AstyanaxOrderedKeyColumnValueStore.java:138)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getSlice(AstyanaxOrderedKeyColumnValueStore.java:88)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration$1.call(KCVSConfiguration.java:70)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration$1.call(KCVSConfiguration.java:64)
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:30)
... 8 more
Caused by: com.netflix.astyanax.connectionpool.exceptions.NoAvailableHostsException: NoAvailableHostsException: [host=None(0.0.0.0):0, latency=0(0), attempts=0]No hosts to borrow from
at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.<init>(RoundRobinExecuteWithFailover.java:30)
at com.netflix.astyanax.connectionpool.impl.TokenAwareConnectionPoolImpl.newExecuteWithFailover(TokenAwareConnectionPoolImpl.java:83)
at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:256)
at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4.execute(ThriftColumnFamilyQueryImpl.java:519)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getNamesSlice(AstyanaxOrderedKeyColumnValueStore.java:136)
... 12 more
Dec 01 2013 15:15:13 [ERROR] main org.slf4j.helpers.MarkerIgnoringBase: Could not successfully complete backend operation due to repeated temporary exceptions after 60000 ms
com.thinkaurelius.titan.core.TitanException: Could not successfully complete backend operation due to repeated temporary exceptions after 60000 ms
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:49)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration.getConfigurationProperty(KCVSConfiguration.java:64)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:268)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1109)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at ch.uzh.ifi.ddis.dm.twhc.graphdb.GraphDbConnectorImpl.connect(GraphDbConnectorImpl.java:43)
at ch.uzh.ifi.ddis.dm.twhc.clusterhierarchy.persistence.BlueprintsSerializer.<init>(BlueprintsSerializer.java:94)
at ch.uzh.ifi.ddis.dm.twhc.input.TestDriver.main(TestDriver.java:166)
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryStorageException: Temporary failure in storage backend
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getNamesSlice(AstyanaxOrderedKeyColumnValueStore.java:138)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getSlice(AstyanaxOrderedKeyColumnValueStore.java:88)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration$1.call(KCVSConfiguration.java:70)
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration$1.call(KCVSConfiguration.java:64)
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:30)
... 8 more
Caused by: com.netflix.astyanax.connectionpool.exceptions.NoAvailableHostsException: NoAvailableHostsException: [host=None(0.0.0.0):0, latency=0(0), attempts=0]No hosts to borrow from
at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.<init>(RoundRobinExecuteWithFailover.java:30)
at com.netflix.astyanax.connectionpool.impl.TokenAwareConnectionPoolImpl.newExecuteWithFailover(TokenAwareConnectionPoolImpl.java:83)
at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:256)
at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4.execute(ThriftColumnFamilyQueryImpl.java:519)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxOrderedKeyColumnValueStore.getNamesSlice(AstyanaxOrderedKeyColumnValueStore.java:136)
... 12 more
Application terminated prematurely!

On my local set up the application runs fine hence I assume it is a configuration problem (of Cassandra?).

Do you have an idea what could cause this connection problem?

Thanks for your hints and happy Sunday,

Best Faber




Matthias Broecheler

unread,
Dec 2, 2013, 2:11:50 PM12/2/13
to aureliu...@googlegroups.com
Hi Faber,

can you

1) try to disable authentication and see if it works then
2) use cassandrathrift as the storage.backend

Thanks,
Matthias


--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.



--
Matthias Broecheler
http://www.matthiasb.com

Faber Chri

unread,
Dec 2, 2013, 6:30:42 PM12/2/13
to aureliu...@googlegroups.com
Hi Matthias

try to disable authentication and see if it works then

Sorry, that's currently not possible. 

use cassandrathrift as the storage.backend

This worked! App runs fine now. Great!
Changed the option storage.backend from "cassandra" to "cassandrathrift".

Your support is very much appreciated.

Best Fabian
Reply all
Reply to author
Forward
0 new messages