New to Kryo, Need Help Deserializing an ArrayList<T>

[Daniel]

Hello,

I'm fairly new to Kryo (and java) and I'm facing the following issue.

I'm trying to deserialize an ArrayList of objects I'm getting as a byte array from a zeromq socket.

The objects at the server and client are the same, i.e. they use the same class library. The server and the client use the same version of Kryo (2.22-SNAPSHOT):

// Server

void selectSomeClassRecords(ZMQ.Socket socket)

{
...
     // query the DB
     Selects selects = new Selects();
     List<SomeClass> someClassRecords = (ArrayList<SomeClass>)selects.selectSomeClassRecords();
     kryo.register(SomeClass.class);
     kryo.register(someClassRecords.getClass());
     kryo.writeObject(output, someClassRecords);
     kryo.reset();
          output.flush();
     byte[] msg = ((ByteArrayOutputStream)output.getOutputStream()).toByteArray();
     output.close();
     socket.send(msg, 0);
...
}

// Client

private void querySomeClass(ZMQ.Socket socket)
{
...
    try
    {
        // receive the buffer
        byte[] buffer = socket.recv(0);
        // restore the buffer into an object
        List<SomeClass> someClassRecords = null;

        kryo.register(SomeClass.class);
        kryo.register(someClassRecords.getClass());       
        Input input = new Input(new ByteArrayInputStream(buffer), buffer.length);
   
        kryo.register(SomeClass.class);
        kryo.register(someClassRecords.getClass());
        someClassRecords = new ArrayList<StudyDesc>(kryo.readObject(input, someClassRecords.getClass()));
        kryo.reset();
    }
    catch (Exception e)
    {
        e.printStackTrace();
    }

    i.close();
...
}

"SomeClass" itself contains references to other classes, some of which contain generic ArrayLists. All the objects in the hierarchy and in the collections are plain java objects and may inherit from other simple classes, all in all is about 8 to 10 levels deep.

The buffer I get back is around 3K and if I make it into a String, just to inspect it, it seems to contain the information I expect, however when I tried to deserialize the buffer with the above query method I get:

com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 12
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
        at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:767)
        at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
        at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:1)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)
        at com.neb.test.App.querySomeClass(App.java:10296)
        at com.neb.test.App.run(App.java:14128)
        at com.neb.test.App.main(App.java:14139)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
        at java.lang.Thread.run(Thread.java:722)

I have Kryo#setAutoRest(false) in both client and server, that is the only explicit setting I have.

I'd appreciate any pointers you could give me on what I'm missing/doing wrong or on ways I could get a more detailed error output (e.g. how to know which one is class ID 12?)

Thanks and best regards,

Daniel.

[mongonix]

On Friday, September 13, 2013 8:12:36 AM UTC+2, Daniel wrote:

Hello,

I'm fairly new to Kryo (and java) and I'm facing the following issue.

I'm trying to deserialize an ArrayList of objects I'm getting as a byte array from a zeromq socket.

The objects at the server and client are the same, i.e. they use the same class library. The server and the client use the same version of Kryo (2.22-SNAPSHOT):

// Server

void selectSomeClassRecords(ZMQ.Socket socket)

{
...
     // query the DB
     Selects selects = new Selects();
     List<SomeClass> someClassRecords = (ArrayList<SomeClass>)selects.selectSomeClassRecords();
     kryo.register(SomeClass.class);
     kryo.register(someClassRecords.getClass());

There is no need to call register every time you perform a (de)serialization. It is enough to do it once when you crate the Kryo instance.

 

     kryo.writeObject(output, someClassRecords);
     kryo.reset();
          output.flush();
     byte[] msg = ((ByteArrayOutputStream)output.getOutputStream()).toByteArray();
     output.close();
     socket.send(msg, 0);
...
}

// Client

private void querySomeClass(ZMQ.Socket socket)
{
...
    try
    {
        // receive the buffer
        byte[] buffer = socket.recv(0);
        // restore the buffer into an object
        List<SomeClass> someClassRecords = null;

        kryo.register(SomeClass.class);
        kryo.register(someClassRecords.getClass());

Same comment here: Don't register classes every time you want to deserialize. Do it only when you create a Kryo instance.

 

       
        Input input = new Input(new ByteArrayInputStream(buffer), buffer.length);

This works. But IIRC there is a constructor for Input, which takes your byte[] as argument directly. It is probably easier to use in this case.

 

   
        kryo.register(SomeClass.class);
        kryo.register(someClassRecords.getClass());

Why do you register your classes again here? You did it just a few lines ago.

 

        someClassRecords = new ArrayList<StudyDesc>(kryo.readObject(input, someClassRecords.getClass()));

This line is wrong. You have written someClassRecords before, which is of type someClassRecords.getClass(), right? So, you should read it in the same way, i.e.

someClassRecords = (ArrayList<StudyDesc>)(kryo.readObject(input, someClassRecords.getClass()));

Or do you try to construct a list of StudyDesc from a list of SomeClass, which is read by kryo.readObject(input, someClassRecords.getClass()?

BTW before this line someClassRecords is not initialized yet, i.e. it is null. Therefore, someClassRecords.getClass() is .... well, it should throw an NPE. A safer bet could be something like (assuming that the real type is ArrayList):

someClassRecords = (ArrayList<StudyDesc>)(kryo.readObject(input, ArrayList.class));

Even safer would be to do:

kryo.writeClassAndObject(output, someClassRecords);

someClassRecords = (List<StudyDesc>)(kryo.readClassAndObject(input));

With this approach a real class will be always written into a serialized stream.

 

        kryo.reset();
    }
    catch (Exception e)
    {
        e.printStackTrace();
    }

    i.close();
...
}

"SomeClass" itself contains references to other classes, some of which contain generic ArrayLists. All the objects in the hierarchy and in the collections are plain java objects and may inherit from other simple classes, all in all is about 8 to 10 levels deep.

The buffer I get back is around 3K and if I make it into a String, just to inspect it, it seems to contain the information I expect, however when I tried to deserialize the buffer with the above query method I get:

com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 12
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
        at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:767)
        at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
        at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:1)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)
        at com.neb.test.App.querySomeClass(App.java:10296)
        at com.neb.test.App.run(App.java:14128)
        at com.neb.test.App.main(App.java:14139)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
        at java.lang.Thread.run(Thread.java:722)

I have Kryo#setAutoRest(false) in both client and server, that is the only explicit setting I have.

May I ask why you're doing it? It is very unusual.

 

I'd appreciate any pointers you could give me on what I'm missing/doing wrong or on ways I could get a more detailed error output (e.g. how to know which one is class ID 12?)

For debug output, you can enable the TRACE level in the logger (see the project home page for details).

 

-Leo 

[Daniel]

Hi Leo,

My issue was, as you pointed out, registering the classes more than once.

Thank you very much for the tips and detail information.

Regards,

Daniel.