Tinkerpop 3.1 and Titan 1.0 Subgraph Gryo Serialization

215 views
Skip to first unread message

Sebastian

unread,
Sep 28, 2015, 10:32:17 PM9/28/15
to Gremlin-users
Hi, I'm taking a look at the new TinkerIoRegistry class in Tinkerpop 3.1, using Titan 1.0 as my graph db (rebuilt to use Tinkerpop 3.1).

I'm trying to serialize and send the result of a subgraph() gremlin operation from my gremlin server to my client. Right before serialization, the result of the subgraph operation is stored in a TinkerGraph object, however, it still contains a few elements of Titan (the RelationIdentifier's). The relation identifier class has already been registered with Kryo at startup using the IoRegistry interfaces.

However, I noticed that the TinkerIoRegistry creates a new instance of the GryoMapper and Kryo objects which do not include these registered classes, it then uses these new instances to try to serialize my graph, but fails when it encounters the unregistered Titan class RelationIdentifier.

Is this expected behavior? 

Thanks,

Sebastian




Stephen Mallette

unread,
Sep 29, 2015, 7:21:24 AM9/29/15
to Gremlin-users
This seems like a valid problem.  I re-opened the issue in JIRA as this isn't implemented properly:


Seems like you dug into the code a bit - do you have any solutions in mind?

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5e91ecdb-409e-4cd3-9888-eeadbc6b68d8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Sebastian

unread,
Sep 30, 2015, 3:04:59 PM9/30/15
to Gremlin-users
One solution that we came up with was to change the GryoIO, GryoReader, GryoWriter class constructors to take in a Kryo instance instance of the GryoMapper. The respective builders were then also changed by adding a method to set a Kryo instance, and finally, changing the Builder.create method to always use the kryo instance UNLESS it's null, in that case do as currently done and just create a new GryoMapper and get the resulting Kryo instance from it.

With those changes we were then able to serialize/deserialize our Titan-backed Tinkergraph subgraphs from client to server, except for some queries which contained other Titan classes (Geoshape.class) which don't seem to be registered with the TitanIORegistry yet. 

i submitted a pull request so you can take a look at it

Stephen Mallette

unread,
Sep 30, 2015, 3:43:51 PM9/30/15
to Gremlin-users
First of all, thanks for digging into this and offering the PR.  I think you're close to a solution.  I believe we can do it though without a lot of rippling change in constructors and builders.  Check out this branch i just added:


I made it so that the various reader/writers that take a Mapper<> don't take concrete implementations.  They instead take the interface Mapper<>....here's the commit with the diff so you can see it clearly:


What's that do?  well, now TinkerGraph could create a class called TinkerGryoMapper that implements Mapper<Kryo>.  TinkerGryoMapper could be constructed with a Kryo instance.  So here:


you would create a TinkerGryoMapper instance, give it the Kryo instance handed to the serializer, then build up a GryoReader by hand like:

GraphSONReader.build().mapper(new TinkerGryoMapper(kryo)).create().readGraph(....)

see how that works?  if that makes sense to you, could you submit a pull request against that branch with appropriate test coverage?  









To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5fd94d59-e6cb-41af-80d5-4d1d7418b591%40googlegroups.com.

Sebastian

unread,
Oct 2, 2015, 10:18:03 AM10/2/15
to Gremlin-users
Yep no problem, that makes sense and seems much cleaner, i'll work off that

Thanks

Stephen Mallette

unread,
Oct 2, 2015, 10:34:50 AM10/2/15
to Gremlin-users
Excellent - I'll keep an eye out for your pull request.  Thanks for handling that.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/b594fe82-bed9-4e1c-b569-6a260a305853%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages