[Tinkerpop 3] Problem with custom class Serialization

[MERAL Guillaume]

Hi everyone,

I'm currently working on my own implementation of Tinkerpop3 Graph.

 I would like to be able to serialize my graphs to be able to use the :remote as objects feature of the Gremlin console.

I first tried to register my custom class in a static bloc like this : 

public class TulipGraph implements Graph , KryoSerializable{

static {

Kryo kryo = new Kryo();

kryo.register(fr.labri.tulipgraph.structure.TulipGraph.class);

}

...

Even though i make sure that this static bloc is executed before i request a graph, i got this error :

com.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Class is not registered: fr.labri.tulipgraph.structure.TulipGraph

I figured out that class registration is not persistent between two instanciations of a kryo object.

How could i configure the kryo instance used by the gremlin server and the gremlin console to be able to register my custom classes ?

Thanks a lot,

Guillaume.

[Stephen Mallette]

Guillaume, thanks for taking the leap and diving into TinkerPop3.  Your early feedback will be very helpful to us.  

You should be able to configure the serializers in the gremlin-server.yaml file.  Note the "serializers" key that has entries like this:

  - { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0 }

That map can also take an optional "config" key, which must also be a map:

  - { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0, config: {...}}

The config will different from one serializer to the next, but for Kryo, it will accept a map that contains a key called "custom" which is a list of fully qualified class names that should be plugged into kryo so something like:

  - { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0, config: {custom:[fr.labri.tulipgraph.structure.TulipGraph]}}

There are two options for registering a "custom" class.  you can provide the class name as above, or you can register the class and the serializer for that class separately.  in that case, you will want to provide a semi-colon separated string to the item in the "custom" list - like:

  - { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0, config: {custom:[fr.labri.tulipgraph.structure.TulipGraph;fr.labri.tulipgraph.structure.TulipGraphSerializer]}}

Best regards,

Stephen

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[MERAL Guillaume]

Great. I no longer have this problem when i request a tulipGraph from the console but server-side only. This exception is still raised client-side : 

 

 com.tinkerpop.gremlin.driver.ser.SerializationException: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 65536

I guess that i now have to do a similar operation in order to register my class for the console deserializer but i could not find any configuration file.

Could you explain me how i should proceed ?

Thanks again,

G.M

[Stephen Mallette]

You've gotten a bit ahead of me as the capability to do custom serialization from the Console wasn't yet available.  :)  You could have done it with gremlin-driver but being able to do it from the Console is an important part of what using Gremlin as a primary tool for analyzing graphs is about.  Anyway, it was an easy feature to add, so I quickly coded it in.  You will need to pull the latest from the TP3 repo and build again to try out the change.  Here's how you do it:

gremlin> serializer = new KryoMessageSerializerV1d0()

==>com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0@6cfcd46d

gremlin> serializer.configure([serializeResultToString: "true"])

==>null

gremlin> :remote as custom serializer

==>results as custom

gremlin> :rem

==>remote - results as custom [localhost/127.0.0.1:8182]

gremlin> :> 1+1

==>Item{resultItem=2 class=java.lang.String}

So basically, you instantiate a new MessageSerializer instance.  That can be your own or one of the provided implementations.  The key is that the MessageSerializer used here must match the configuration of the one on the server.  You should already be aware of the configure() method on the MessageSerializer...use that to add your custom classes.  You can make those available by copying your custom jar to the Console path or trying the :use function to pull in the dependency.  

The new piece I added to the Console comes next.  You do:

gremlin> :remote as custom serializer

"custom" is a new parameter to the :remote command, which takes a second argument representing the name of the variable that contains the MessageSerializer instance.  In the example above, I just configured the "serializer" to do the same as if you had done ":remote as text".

Thanks for digging into this RFClease of TP3 and trying out these features.  Your feedback and findings will be very helpful in solidifying the feature set and codebase.

Stephen

[Alexandre Perrot]

Hi,

I'm working with Guillaume on this custom Graph implementation.
Thank you for your light-speed support and feedback :)

The way you register classes for custom serialization in Tinkerpop3 seems a bit too heavy on configuration.
The user would have to configure the configuration files on server side (this part should be done only once, so this is not my major concern), and on client side configure manually via the console every custom serializer in the same order as on the server side every time he connects to the server ?

Wouldn't it be simpler to register classes via code when a jar is loaded ?

[Stephen Mallette]

Since "order" is important to Kryo, I think it's important that this process remain in the control of the user.  I guess I'm thinking that automating that load might accidentally create conflicts that would prevent things from working properly.  That said, I can imagine that manual configuration, especially in the Console would be tedious.

Here's two ways you could handle it:

1. Like TP2, you can provide a initialization script to the Gremlin Console.  In that script you could initialize your serializer.  Just be sure to start the console with:

$bin/gremlin.sh init.groovy

2. Note the features of :remote:

gremlin> :? :remote

usage: :remote [current|connect [<host>|<configuration-file>]|as [text|objects]|timeout <milliseconds>]

You can supply :remote a configuration file.  While that feature has been there, it hasn't dealt with serializer configuration so well, so I made some quick modifications (you will need to pull and rebuild).  You can now do this:

gremlin> :remote connect config/remote.yaml

==>connected - localhost/127.0.0.1:8182

gremlin> :> 1+1

==>Item{resultItem=2 class=java.lang.String}

where remote.yaml looks like this:

https://github.com/tinkerpop/tinkerpop3/blob/4c7297c530083d58735fca52fe89210f674f9375/gremlin-groovy/src/main/config/remote.yaml

Hopefully those options make things easier for you.  Thanks again for the feedback.

Stephen

[MERAL Guillaume]

Hi,

I tried again to serialize my custom class but ran into a new problem.

This is my current state for the problem :

Server side : I have the right configuration file including this element in the serializers list :

- { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0, config: {custom:[fr.labri.tulipgraph.structure.TulipGraph;fr.labri.tulipgraph.structure.TulipGraphSerializer]}}

The serialization part seems ok.

Console side : I saw that you created a new module for the gremlin-console, but it seems that the :remote command has changed a bit.

I created the following remote.yaml file in the config/ directory with the following contents :

hosts: [localhost]                                                                                                                                                                                                                                                           

port: 8182                                                                                                                                                                                                                                                                      

serializer: { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0,config: {custom:[fr.labri.tulipgraph.structure.TulipGraph;fr.labri.tulipgraph.structure.TulipGraphSerializer]}}

I now connect to the server with these commands to retreive my TulipGraph from the server :

gremlin> :remote connect server config/remote.yaml

gremlin> :remote config as objects                         

Since i don't get any error or message when the name of the configuration file is incorrect i guess that i'm not writing the commands properly.

Therefore i tried to get documentation for the remote command just like you showed in your previous post but got this error :

gremlin> :? :remote                                                                                                                              

Can't find bundle for base name com.tinkerpop.gremlin.console.commands.RemoteCommand, locale fr_FR

Display stack trace? [yN]                                                                                                                      

How get to know the right syntax ?

Thanks in advance for your help.

Best regards,

Guillaume

[Stephen Mallette]

Looks like there was a bug in the code. Apparently after years of programming I can still mess up simple logical conditions - sigh.  Please pull latest, rebuild, and see if that change solves your problem.

As for the help command problem - that's something else.  I don't know why that happens other than we don't have a french resource bundle for the help text, though I would have expected it to fall back to english and not present an error.  That's technically a Groovy Console "feature".  If you ever want to get at the actual text for the commands they are here:

https://github.com/tinkerpop/tinkerpop3/tree/master/gremlin-console/src/main/resources/com/tinkerpop/gremlin/console

Stephen

[MERAL Guillaume]

Indeed i was giving the connect command a wrong path for my configuration file. Even though this path seems correct now, the following exception is raised when i request the g graph via submit :

gremlin> :> g

WARN  com.tinkerpop.gremlin.driver.MessageSerializer  - Response [UnpooledUnsafeDirectByteBuf(ridx: 46, widx: 46, cap: 46)] could not be deserialized by com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0.

WARN  io.netty.channel.DefaultChannelPipeline  - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

com.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map

at com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0.deserializeResponse(KryoMessageSerializerV1d0.java:152)

at com.tinkerpop.gremlin.driver.Handler$GremlinResponseDecoder.channelRead0(Handler.java:121)

at com.tinkerpop.gremlin.driver.Handler$GremlinResponseDecoder.channelRead0(Handler.java:105)

at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)

at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:341)

at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:327)

at com.tinkerpop.gremlin.driver.Handler$WebSocketClientHandler.channelRead0(Handler.java:87)

at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)

at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:341)

at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:327)

at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)

at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:341)

at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:327)

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)

at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:341)

at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:327)

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:126)

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)

at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map

at com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0.deserializeResponse(KryoMessageSerializerV1d0.java:143)

... 23 more

The TulipGraphSerializer.read method doesn't even seem to be called.

I probably missed something...

[Stephen Mallette]

Well, a bad path would definitely be a problem, but I also definitely had a bug in my code.  Did you actually pull my minor fix and rebuild to get this error?

--

[MERAL Guillaume]

Yes i did.

If i give a wrong path for the configuration file, i now have this error as expected :

gremlin> :remote connect server remotote.yaml

==>the 'connect' option must be a resolvable host or a configuration file

[Stephen Mallette]

That's good.  Thanks for confirming.  I'm going to guess that you have a configuration problem between your yaml file and Gremlin Server.  If they don't align well you will have these kinds of serialization problems.  Kryo isn't always very nice about the specific nature of errors so you usually get some obtuse looking error like this one.  Do you mind sharing your gremlin-server yaml, remote yaml, and the exact sequence of gremlin console steps you are taking.  Perhaps they are all here in this thread, but i'm not sure what has changed at this point, so it might be useful to re-group a bit on what is current.  Thanks.

[MERAL Guillaume]

I first start in a terminal the gremlin-server from the root directory of tinkerpop3

In another terminal i launch the gremlin-console (remote.yaml being in the root directory of tinkerpop3 too)

[gmeral@gmeral-labri tinkerpop3]$ ./bin/gremlin.sh 

                                                                                                                                                                             

         \,,,/                                                                                                                                                              

         (o o)                                                                                                                                                            

─────oOOo-(3)-oOOo─────                                                                                                                               

gremlin> :remote connect server remote.yaml                                                                                                          

==>connected - localhost/127.0.0.1:8182                                                                                                                 

gremlin> :remote config as objects                                                                                                                          

==>gremlin server - results as objects                                                                                                                     

gremlin> :> g                                                                                                                                                            

WARN  com.tinkerpop.gremlin.driver.MessageSerializer  - Response [UnpooledUnsafeDirectByteBuf(ridx: 46, widx: 46, cap: 46)] could not be deserialized by com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0.

WARN  io.netty.channel.DefaultChannelPipeline  - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

com.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map

...                                                                                                                                                            

Server side i got this, as expected : 

[INFO] GremlinServer - 

         \,,,/

         (o o)

-----oOOo-(_)-oOOo-----

[INFO] GremlinServer - Configuring Gremlin Server from config/gremlin-server.yamll                                                                                                                               

[INFO] MetricManager - Configured Metrics ConsoleReporter configured with report interval=180000msl                                                                                                   

[INFO] MetricManager - Configured Metrics CsvReporter configured with report interval=180000ms to fileName=/tmp/gremlin-server-metrics.csvl                                       

[INFO] MetricManager - Configured Metrics JmxReporter configured with domain= and agentId=l                                                                                                            

[INFO] MetricManager - Configured Metrics Slf4jReporter configured with interval=180000ms and loggerName=com.tinkerpop.gremlin.server.Settings$Slf4jReporterMetrics

[INFO] MetricManager - Configured Ganglia Metrics reporter host=localhost interval=180000ms port=8649 addrmode=MULTICAST ttl=1 proto31=true uuid=null spoof=null

[INFO] MetricManager - Configured Graphite reporter host=localhost interval=180000ms port=2003 prefix=l                                                                                            

[INFO] GremlinServer$WebSocketServerInitializer - Configured application/vnd.gremlin-v1.0+kryo with com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0

[INFO] GremlinServer$WebSocketServerInitializer - Configured application/vnd.gremlin-v1.0+kryo-stringd with com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0

[INFO] GremlinServer$WebSocketServerInitializer - Configured application/vnd.gremlin-v1.0+json with com.tinkerpop.gremlin.driver.ser.JsonMessageSerializerGremlinV1d0

[INFO] GremlinServer$WebSocketServerInitializer - Configured application/vnd.gremlin-v1.0+kryo with com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0

[INFO] Graphs - Graph [g] was successfully configured via [config/tulipgraph-empty.properties].l                                                                                                  

[INFO] GremlinServer$WebSocketServerInitializer - Initialized Gremlin thread pool.  Threads in pool named with pattern gremlin-*l                                                   

[INFO] GremlinExecutor - Getting dependencies for [[org.apache.commons, commons-math3, 3.2]]l                                                                                              

[INFO] GremlinServer$WebSocketServerInitializer - Initialized GremlinExecutor and configured ScriptEngines.l                                                                              

[INFO] GremlinServer - Gremlin Server configured with worker thread pool of 1 and boss thread pool of 1l                                                                                      

[INFO] GremlinServer - Websocket channel started at port 8182.l                                                                                                                                                

[INFO] OpLoader - Adding the standard OpProcessor.l                                                                                                                                                                

[INFO] OpLoader - Adding the control OpProcessor.l                                                                                                                                                                   

TulipGraph Serialization Successfull                                                                                                                                                                                          

[Stephen Mallette]

Not sure what's wrong based on what you sent.  Interesting that you have 4 configured serializers, but only 3 in your gremlin-server.yaml.  Not sure what that means.  Are you sure you are using the "correct" yaml file?  

Note the first line from Gremlin Server logs:

[INFO] GremlinServer - Configuring Gremlin Server from config/gremlin-server.yamll      

That file has a double "l" at the end: "yamll".  The one you sent me only has one "l".  Could that be an issue?

--

[MERAL Guillaume]

Using the find command on the tinkerpop3 directory, i can tell that there is no gremlin-server.yamll (with two l) in it, so it must be a typo error when i typed my message.

As for the serializers, i just mistakenly attached the wrong gremlin-server.yaml file (the default one that is in gremlin-console/config) instead of the one i actually use ton configure the server.

Here is the actual file i use (this time from tinkerpop3/gremlin-server/target/gremlin-server-3.0.0-SNAPSHOT-standalone/config/).

Sorry for this.

[Stephen Mallette]

What happens if you remove the first configuration of the KryoMessageSerializerV1d0?  Does it work then?

--

[MERAL Guillaume]

No, i got the exact same error when i only keep these serializers in the gremlin-server.yaml file : 

  - { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0, config: { serializeResultToString: true }}

  - { className: com.tinkerpop.gremlin.driver.ser.JsonMessageSerializerGremlinV1d0 }

  - { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0, config: {custom:[fr.labri.tulipgraph.structure.TulipGraph;fr.labri.tulipgraph.structure.TulipGraphSerializer]}}

I also tried with only my custom serializer : 

  - { className: com.tinkerpop.gremlin.driver.ser.KryoMessageSerializerV1d0, config: {custom:[fr.labri.tulipgraph.structure.TulipGraph;fr.labri.tulipgraph.structure.TulipGraphSerializer]}}

and the error appears again.

[Stephen Mallette]

Ok, that makes sense actually.  Technically the serializers lower in the configuration were replacing the ones higher in the stack, but I wasn't sure if that was processing out of order.  I've just pushed a quick change that prevents that from happening - now you get a warning message.  

That said, I think I might know the problem now.  In short, don't issue this command:

:remote config as objects

I think that's just overriding the serializer config you provided in your remote.yaml and using the default serializer built into the Console.  Let's see if that solves the problem....

Stephen

[MERAL Guillaume]

I got this :

gremlin> :remote connect server remote.yaml                                         

==>connected - localhost/127.0.0.1:8182                                                

gremlin> :> g                                                                                        

==>Item{resultItem=tulipgraph[vertices:0 edges:0] class=java.lang.String}

Unfortunately this is the same result that i have if don't feed the :remote command any configuration file.

It feels like the remote.yaml file is completely ignored by :remote

[Stephen Mallette]

Once again, I think I know the problem, however let's please take our discussion off-list in case this drags on further:

https://github.com/tinkerpop/tinkerpop3/issues/16

Thanks for continuing to help sort this out.

[MERAL Guillaume]

I agree with this.

On top of this, i find that the :remote command syntax starts to get really confusing.

I replied your issue report on github.