org.apache.tinkerpop.shaded.kryo.KryoException: Encountered unregistered class ID: 65536

[SN]

I am new to titan and trying simple Kyro input and output from the same Titan instance.

I am doing

graph = TinkerFactory.createModern(); graph.io(IoCore.gryo()).writeGraph("tinkerpop-modern.kryo");

And then I am doing

newGraph = TinkerGraph.open(); newGraph.io(IoCore.gryo()).readGraph("tinkerpop-modern.kryo");

And I get the bug below.

Since I am trying this within the same gremlin instance running locally on my machine,

I dont know why my Serialization properties be incorrect.

Is there anything I am doing incorrectly here?

java.lang.RuntimeException: org.apache.tinkerpop.shaded.kryo.KryoException: Encountered unregistered class ID: 65536

at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader$VertexInputIterator.next(GryoReader.java:309)

at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader$VertexInputIterator.next(GryoReader.java:283)

at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.iterate(IteratorUtils.java:70)

at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader.readGraph(GryoReader.java:90)

at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoIo.readGraph(GryoIo.java:86)

at org.apache.tinkerpop.gremlin.structure.io.Io$readGraph.call(Unknown Source)

at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:110)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:122)

at groovysh_evaluate.run(groovysh_evaluate:3)

at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)

at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:69)

at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:185)

at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:119)

at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:94)

at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)

at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)

at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1207)

at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:130)

at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:150)

at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:123)

at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:58)

at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)

at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)

at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1207)

at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:130)

at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:150)

at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:82)

at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)

at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:144)

at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)

at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:303)

Caused by: org.apache.tinkerpop.shaded.kryo.KryoException: Encountered unregistered class ID: 65536

at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoClassResolver.readClass(GryoClassResolver.java:171)

at org.apache.tinkerpop.shaded.kryo.Kryo.readClass(Kryo.java:667)

at org.apache.tinkerpop.shaded.kryo.Kryo.readClassAndObject(Kryo.java:778)

at org.apache.tinkerpop.gremlin.structure.util.star.StarGraphGryoSerializer.readEdges(StarGraphGryoSerializer.java:148)

at org.apache.tinkerpop.gremlin.structure.util.star.StarGraphGryoSerializer.read(StarGraphGryoSerializer.java:105)

at org.apache.tinkerpop.gremlin.structure.util.star.StarGraphGryoSerializer.read(StarGraphGryoSerializer.java:47)

at org.apache.tinkerpop.shaded.kryo.Kryo.readObject(Kryo.java:683)

at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader.readVertexInternal(GryoReader.java:227)

at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader.access$200(GryoReader.java:61)

at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader$VertexInputIterator.next(GryoReader.java:307)

[Daniel Kuppitz]

Can you please show the full Gremlin console session? I just tried it out and it all worked as expected:

gremlin> graph = TinkerFactory.createModern()
==>tinkergraph[vertices:6 edges:6]
gremlin> graph.io(IoCore.gryo()).writeGraph("tinkerpop-modern.kryo")
==>null
gremlin> newGraph = TinkerGraph.open()
==>tinkergraph[vertices:0 edges:0]
gremlin> newGraph.io(IoCore.gryo()).readGraph("tinkerpop-modern.kryo")
==>null
gremlin> newGraph
==>tinkergraph[vertices:6 edges:6]

Also, which version are you using?

And as an aside: TinkerGraph is not Titan.

Cheers,

Daniel

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aureliusgraphs/886d9fdd-0cc7-4935-ad64-61fce6a58473%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[SN]

Hi Daniel,

I am able to get it running now if I do it with the modern.

gremlin> graph = TinkerFactory.createModern();

==>tinkergraph[vertices:6 edges:6]

gremlin> graph.io(IoCore.gryo()).writeGraph("tinkerpop-modern.kryo")

==>null

gremlin> newGraph = TinkerGraph.open()

==>tinkergraph[vertices:0 edges:0]

gremlin> newGraph.io(IoCore.gryo()).readGraph("tinkerpop-modern.kryo")

==>null

gremlin> newGraph

==>tinkergraph[vertices:6 edges:6]

gremlin> Gremlin.version

==>3.0.1-incubating

gremlin>

But when I try to do it with actual data in the graph, I get a similar error:

gremlin> graph = TitanFactory.open('conf/titan.properties')

==>standardtitangraph[cassandra:[localhost]]

gremlin> graph.io(IoCore.gryo()).writeGraph("tinkerpop-backup.kryo")

==>null

gremlin> newGraph = TinkerGraph.open();

==>tinkergraph[vertices:0 edges:0]

gremlin> newGraph.io(IoCore.gryo()).readGraph("tinkerpop-backup.kryo")

org.apache.tinkerpop.shaded.kryo.KryoException: Encountered unregistered class ID: 65536

Display stack trace? [yN] y

java.lang.RuntimeException: org.apache.tinkerpop.shaded.kryo.KryoException: Encountered unregistered class ID: 65536

I am wondering what I need to do to get past this exception and also once I am able to read data in newGraph, how do I commit it to another instance?

Because I tried it with the tinkerpop-modern where I am able to read the data in the newGraph but not sure how to push that to instance.

Thanks,

SN

[Daniel Kuppitz]

Hmm, I have no idea about the exception. But can you talk more about what you're actually trying to accomplish? It's not clear to me, why you push data back and  forth between 2 graph implementations. Is the ultimate goal to copy a graph from one Titan cluster to another?

Cheers,

Daniel

--

You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/aureliusgraphs/fc7a24c2-006e-489e-99b0-42ed6b7ff52c%40googlegroups.com.

[Jason Plurad]

Hi,

Daniel was spot on yesterday: "And as an aside: TinkerGraph is not Titan." The exception is because a TITAN graph was serialized as gryo, and then attempted to import into a TINKER graph.

Titan uses a custom serializer for edge ids, and since Gryo was chosen as the output format, the id class type is encoded in the gryo file. If you needed to import that gryo file into TinkerGraph, you'd have to register the edge serializer class. You can get a clean run by using two TitanGraph instances on the write and the read, since TitanGraph will use the same serializer, i.e.:

newGraph = TitanFactory.open('inmemory') // or a properties file referring to the new location

Another option would be to use a lossy format like GraphSON or GraphML which can serialize the edge ids as Strings.

Daniel's questions still stand... "But can you talk more about what you're actually trying to accomplish? It's not clear to me, why you push data back and  forth between 2 (different) graph implementations. Is the ultimate goal to copy a graph from one Titan cluster to another?"

-- Jason

[San]

Hi Guys,

Yes my goal is to copy Data from one instance to another.

And I was hoping to use the non lossy format and use Kryo for that.

Thanks,

San

[Daniel Kuppitz]

Then, if your graph is small enough, you can just write it out as a Gryo file (as you already did) and then load it back into another Titan (!) graph.

This approach will fail at some point, when your graph gets too large. In that case you can dump your graph using BulkDumperVertexProgram and load it back using BulkLoaderVertexProgram.

Cheers,

Daniel

To view this discussion on the web visit https://groups.google.com/d/msgid/aureliusgraphs/f66bbf69-c6c7-4d5a-a05e-cb6d6fdad521%40googlegroups.com.

[Sachin]

Thats what I was trying to figure how to get the Gryo file working.

I created the file using the following commands 

gremlin> graph = TitanFactory.open('conf/titan.properties')

==>standardtitangraph[cassandra:[localhost]]

gremlin> graph.io(IoCore.gryo()).writeGraph("tinkerpop-backup.kryo")

==>null

But Since I am using Titan to create the graph, I get the Kryo exception when loading it back.

I am trying to figure what is the right way to create the the kryo file.

The size of our database is decent enough not too small. Infact we are facing the frame size errors as well as I have mentioned in another post.

The problem with BulkLoaderVertexProgram is it doesnt support multi valued properties which we are using.

But I guess with Gryo not going to work eventually, we will need to use BulkLoaderVertexProgram with a custom implementation eventually.

Just as a thought, is it possible somehow to get specific data files from underneath cassandra instance and port them over to another instance.

Would that work?

Thanks for the inputs.

Thanks,

Sachin

[Daniel Kuppitz]

Again, the failure case you were showing was odd; you've tried to load your data back into a TinkerGraph, not a TitanGraph. Are you saying that you face the same problem when you try to load the Gryo file back into another TitanGraph?

Cheers,

Daniel

To view this discussion on the web visit https://groups.google.com/d/msgid/aureliusgraphs/12e7cca7-6224-4479-8cd4-734452ba7253%40googlegroups.com.

[Sachin]

I think that was the mistake that I was doing.

While reading back I should have done 

newGraph = TitanFactory.open('conf/titan.properties')

instead of 

newGraph = TinkerGraph.open();

and then 

 newGraph.io(IoCore.gryo()).readGraph("tinkerpop-backup.kryo")

Atleast I am able to get it running locally now.

Hopefully it works in the production db else will need to go with BulkLoaderProgram.

Thanks a lot for all the inputs.

Thanks,

Sachin

[jun yang]

Hi Jason,

I hit the same problem, that we can't write titan graph instance kryo to Tinker/Hadoop graph, and Tinker/Hadoop graph instance kryo can writer to Titan graph, I think it make sense as you mentioned.

I'm wondering how to "register the edge serializer class" ?  it just a configuration or need some implementation in TinkerPop code ?

Thanks!

Jun(Terry) Yang.

[Jason Plurad]

I ran these steps on the titan11 branch, which is using TinkerPop 3.1.1-incubating.

// Serialize a TitanGraph with Gryo
graph = TitanFactory.open('inmemory')
graph.io(gryo()).readGraph('data/tinkerpop-modern.kryo')
graph.io(gryo()).writeGraph('/tmp/modern.kryo')
graph.close()

// Read Gryo into a TitanGraph
graph = TitanFactory.open('inmemory')
graph.io(gryo()).readGraph('/tmp/modern.kryo')
g = graph.traversal()
g.V().valueMap(true)
graph.close()

// Read Gryo into a TinkerGraph
graph = TinkerGraph.open()
gryoIo = IoCore.gryo().graph(graph).registry(TitanIoRegistry.getInstance()).create()
gryoIo.readGraph('/tmp/modern.kryo')
g = graph.traversal()
g.V().valueMap(true)
graph.close()