Configure multiple cassandra instances in titangraph
Hari
Hi,
I want to store my titangraph on multiple cassandra instances. So I started two apache cassandra instances (say cassandra1 & cassandra2) on same system and changed the following configurations in cassandra.yaml:
replaced "localhost" with 127.0.0.1 in cassandra1 and 127.0.0.2 in cassandra2
rpc_port 9160 for cassandra1 and 9161 for cassandra2
seeds: "127.0.0.1,127.0.0.2" in both the copies
JMX_PORT different in both cassandra
Below are the configurations in my titangraph:
1 BaseConfiguration conf = new BaseConfiguration();
2 conf.setProperty("storage.backend", "cassandra");
3 conf.setProperty("storage.hostname", "127.0.0.1,127.0.0.2");
4 TitanGraph tg = new TitanGraph(conf);
...
...
Below is the error at line number 4:
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
java.lang.IllegalArgumentException: Could not instantiate
implementation:
com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager
at com.thinkaurelius.titan.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:55)
at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:473)
at com.thinkaurelius.titan.diskstorage.Backend.getStorageManager(Backend.java:407)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1320)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:94)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:74)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.thinkaurelius.titan.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:44)
... 7 more
Caused by: java.lang.NoSuchMethodError:
com.netflix.astyanax.impl.AstyanaxConfigurationImpl.setTargetCassandraVersion(Ljava/lang/String;)Lcom/netflix/astyanax/impl/AstyanaxConfigurationImpl;
at
com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.getContextBuilder(AstyanaxStoreManager.java:505)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.(AstyanaxStoreManager.java:289)
... 12 more
Can someone please suggest if this is the correct way of configuring the titan graph and cassandra DBs.
Jason Plurad
I'm not clear on what you are trying to accomplish. Are you trying to configure a Cassandra cluster with multiple nodes?
The stack trace you provided seems to indicate an issue in your dependencies, somewhat unrelated to the overall configuration properties.
-- Jason
Hari
Yes I want to configure Cassandra cluster with multiple nodes. Do I need to configure two cassandra instances or can I do this in single instance of Cassandra also ?
-- Hari
Jason Plurad
You should refer to Cassandra documentation on how to configure a multiple node cluster.
http://docs.datastax.com/en/cassandra/2.1/cassandra/initialize/initializeSingleDS.html
Typically in the cassandra.yaml you would update the seeds, listen_address, rpc_address, start_rpc properties.
You should keep the rpc_port the same across the Cassandra nodes because Titan only accepts a single port instead of a list of ports for the storage.port value.
-- Jason
Hari
I configured Cassandra as given in the above doc. I am running 2 nodes in same server so configured rpc_address, listen_address as 127.0.0.1 in node1 and 127.0.02 in node2 and seeds configured as "127.0.0.1,127.0.0.2". Both the nodes are up and running in my server.
./nodetool status results this in node1:
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 127.0.0.2 330.83 KiB 256 48.9% d3663063-40ce-4299-8b14-c0ba4bcdc911 rack1
UN 9.121.59.100 226.87 KiB 256 51.1% 3f3ba9c6-7fbe-4738-a0b1-0bbeba303205 rack1
./nodetool status results this in node2:
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 127.0.0.1 174.88 KiB 256 100.0% d3663063-40ce-4299-8b14-c0ba4bcdc911 rack1
I configured both IPs in my code as below:
1 BaseConfiguration conf = new BaseConfiguration();
3 conf.setProperty("storage.hostname","127.0.0.1,127.0.0.2");
5 tg.io(IoCore.graphson()).readGraph("myTestFile.gson");
There are 2 issues I am facing now with the above setup and code.
Issue1 -> There is no error while running this code for the first time but data is stored only in node. Number of vertices in node1 is >0 but in second node it is showing 0.
Issue 2 -> If I run the same code again to load another gson file, it gives me error saying the below at line5:
21:43:40,776 ERROR StandardTitanGraph:779 - Could not commit transaction [1] due to exception
com.thinkaurelius.titan.core.TitanException: Could not execute operation due to backend exception
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:44)
at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:87)
at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:141)
at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:198)
at com.thinkaurelius.titan.diskstorage.BackendTransaction.commit(BackendTransaction.java:135)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.commit(StandardTitanGraph.java:692)
at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.commit(StandardTitanTx.java:1352)
at com.thinkaurelius.titan.graphdb.tinkerpop.TitanBlueprintsGraph$GraphTransaction.doCommit(TitanBlueprintsGraph.java:263)
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:94)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONReader.lambda$readGraph$55(GraphSONReader.java:106)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONReader.readGraph(GraphSONReader.java:102)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONIo.readGraph(GraphSONIo.java:87)
Caused by: com.thinkaurelius.titan.diskstorage.locking.PermanentLockingException: Expected value mismatch for KeyColumn [k=0x 16-165-160-114-116- 30- 65- 84- 76-198, c=0x 0]: expected=[] vs actual=[ 80-133] (store=graphindex)
at com.thinkaurelius.titan.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkSingleExpectedValueUnsafe(ExpectedValueCheckingTransaction.java:275)
at com.thinkaurelius.titan.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.access$000(ExpectedValueCheckingTransaction.java:36)
at com.thinkaurelius.titan.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction$1.call(ExpectedValueCheckingTransaction.java:213)
at com.thinkaurelius.titan.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction$1.call(ExpectedValueCheckingTransaction.java:210)
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:56)
at com.thinkaurelius.titan.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkSingleExpectedValue(ExpectedValueCheckingTransaction.java:210)
at com.thinkaurelius.titan.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllExpectedValues(ExpectedValueCheckingTransaction.java:179)
at com.thinkaurelius.titan.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:144)
at com.thinkaurelius.titan.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:58)
at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:90)
at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:87)
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:56)
at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:42)
... 23 more
Please let me know if anything is wrong or missing here.
Thanks,
Haritha
Jason Plurad
Based on the results from nodetool, it doesn't look like the set up is configured correctly. I would expect to see the same IPs (127.0.0.1, 127.0.0.2) in both results. Questions for setting up and configuring Cassandra might be better suited on the Cassandra users mailing list.
I'd suggest that you look into either Docker or Vagrant if you're trying to run a multi-node cluster on the same machine. This https://github.com/pcmanus/ccm might be helpful too.
-- Jason
Disruptive Solutions
Op donderdag 27 oktober 2016 16:58:03 UTC+2 schreef Hari: