Running Titan + Tinkerpop 3.2 with AWS DynamoDB

Dave vU

unread,

Oct 30, 2016, 10:38:49 PM10/30/16

to Gremlin-users

Is it currently possible to use Tinkerpop 3.2.3 with AWS DynamoDB?

Following these instructions, I am able to successfully run Titan 1.0 + TinkerPop 3.0-incubating with a DynamoDB storage backend.

But I would like to use the gremlin-python scriptEngine, which I believe requires Tinkerpop 3.2…

This post seems to indicate that Titan 1.0 should be compatible with TP3.2, however I am not sure how I would go about editing the Maven script to do that (very much a Maven newbie here). I tried replacing the Tinkerpop version to point at 3.2.3, but somehow it seems to still install some 3.0 dependencies somewhere, resulting in this error when I try to run the server:

`Caused by: groovy.lang.GroovyRuntimeException: Conflicting module versions. Module [groovy-jsr223 is loaded in version 2.4.1 and you are trying to load version 2.4.7`

Does anyone know if this can be done or if I am better off giving up and waiting for some later version to bring this support?

Alternatively: are there any good options to get Tinkerpop+gremlin-python running on an AWS server with any storage backend?

Thanks in advance!

--

Dave

Václav Pavlín

unread,

Dec 14, 2016, 7:58:34 AM12/14/16

to Gremlin-users

I am trying to achieve the same thing for the last week with no luck so far...

Cheers,

Vašek

Dne pondělí 31. října 2016 3:38:49 UTC+1 Dave vU napsal(a):

Jason Plurad

unread,

Dec 14, 2016, 10:36:53 PM12/14/16

to Gremlin-users

This would be much easier if there were a more recent releases of Titan and DynamoDB-Titan Storage, but it can still be done.

I've posted previously directions on how to build the titan11 with an update to TinkerPop 3.2.x -- currently at TP 3.2.3, thanks to solid contributions from @dylanht and @sjudeng -- https://groups.google.com/d/msg/gremlin-users/ajs0C4-0vzY/SEIDhczrAgAJ

Next if you want to use the DynamoDB backend, you need to make a few changes in your dynamodb-titan-storage-backend repo before building it with mvn install so that it uses the locally-built Titan snapshot. The patch updates the TinkerPop and Titan version and updates the Gremlin Server configs:

git clone https://github.com/awslabs/dynamodb-titan-storage-backend.git
cd dynamodb-titan-storage-backend
curl https://gist.githubusercontent.com/pluradj/d56c1948f4665ee7fb1bc35daeba4f92/raw/be5f639a64c8d6ac196c59eb7e6d1a1903015b17/dynamo-titan11-tp323.patch | git apply -v --index
mvn install
mvn test -Pstart-dynamodb-local

Then before starting the Gremlin Server in step 7, make sure to install the gremlin-python plugin on the server:

src/test/resources/install-gremlin-server.sh
cd server/dynamodb-titan100-storage-backend-1.0.0-hadoop1
bin/gremlin-server.sh -i org.apache.tinkerpop gremlin-python 3.2.3
bin/gremlin-server.sh conf/gremlin-server/gremlin-server-local.yaml

Load the Marvel test data using Gremlin Console:

bin/gremlin.sh
gremlin> :remote connect tinkerpop.server conf/remote.yaml
gremlin> :> com.amazon.titan.example.MarvelGraphFactory.load(graph, 100, false)

Finally use the Python REPL to query the Gremlin Server

python
>>> from gremlin_python import statics
>>> from gremlin_python.structure.graph import Graph
>>> from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
>>> statics.load_statics(globals())
>>> graph = Graph()
>>> g = graph.traversal().withRemote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
>>> g.V().has('weapon', 'claws').valueMap().toList()

Here is the tutorial link for Python + Gremlin Server.

-- Jason

Václav Pavlín

unread,

Dec 16, 2016, 11:23:28 AM12/16/16

to Gremlin-users

Hi Jason,

Thank you for the response. I hope I followed your steps properly, but I am getting this error now:

Caused by: com.thinkaurelius.titan.core.TitanException: StorageBackend version is incompatible with current Titan version: storage=1.0.0 vs. runtime=1.1.0-SNAPSHOT

at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1373)

at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:94)

at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:74)

I am not sure where to look nor where could this come from..any ideas?

Cheers,

Vašek

Dne čtvrtek 15. prosince 2016 4:36:53 UTC+1 Jason Plurad napsal(a):

Jason Plurad

unread,

Dec 16, 2016, 1:40:41 PM12/16/16

to Gremlin-users

When you build the titan11, update the root pom.xml with this:

<titan.compatible.versions>1.0.0</titan.compatible.versions>

It's found right before the TinkerPop version.

-- Jason

Václav Pavlín

unread,

Jan 12, 2017, 10:27:13 AM1/12/17

to Gremlin-users

Thank you Jason, that fixed my issue and I am able to use gemlin-python succesfully with AWS DynamoDB as backend.

For a reference, I've created a Dockerfile using your how to: https://github.com/containscafeine/data-model/ is anyone would find it useful:) (this is a coleague's repo who submitted it to CentOS community build service to have images available in some registry)