Custom Map implementation and KryoSerializable

[Niels Basjes]

Hi,

I have created a custom implementation of the java.util.Map that is to be used by others as a library and I want that to be serializable with Kryo.

The problem I have is that if I define my class to implement KryoSerializable I expect Kryo to use the specified write and read methods for the (de)serialization.

https://github.com/EsotericSoftware/kryo/blob/master/README.md#kryoserializable

The problem I have is that it doesn't : it uses the more generic MapSerializer instead and as a consequence I get NPE.

My question to you: What have I done wrong; or is this a bug in Kryo ?

I have been able to reproduce this effect to this:

( Full reproduction: https://github.com/nielsbasjes/BugReport-KryoMap )

I'm using this dependency in my project:

<dependency>
  <groupId>com.esotericsoftware</groupId>
  <artifactId>kryo</artifactId>
  <version>5.0.0-RC9</version>
  <scope>provided</scope>
</dependency>

and I have defined my Map as (short version)

public class MyCustomMap<V extends Serializable> implements Map<String, V>, KryoSerializable, Serializable {
  private Boolean someFlag;
  private TreeMap<String, V> stuff;

  private MyCustomMap() {

  }

  public MyCustomMap(boolean newSomeFlag) {

    someFlag = newSomeFlag;

    stuff = new TreeMap<>();

  }

  @Override

  public void write(Kryo kryo, Output output) {

    output.writeBoolean(someFlag);

    kryo.writeClassAndObject(output, stuff);

  }

  @Override
  @SuppressWarnings("unchecked")
  public void read(Kryo kryo, Input input) {
    someFlag = input.readBoolean();
    stuff = (TreeMap<String, V>) kryo.readClassAndObject(input);
  }

....

}

Then in a test I do this 

Kryo kryo = new Kryo();
kryo.register(MyCustomMap.class);

ByteBufferOutput byteBufferOutput = new ByteBufferOutput(1_000_000, -1);
kryo.writeClassAndObject(byteBufferOutput, instance);
ByteBuffer bytes = byteBufferOutput.getByteBuffer();

bytes.rewind();
ByteBufferInput byteBufferInput = new ByteBufferInput(bytes);
Object mapObject = kryo.readClassAndObject(byteBufferInput);

This is the stacktrace I get seems to indicate that it isn't using the write/read I specified but the more generic MapSerializer instead.

java.lang.NullPointerException

at nl.basjes....@2.0-SNAPSHOT/nl.basjes.collections.MyCustomMap.put(MyCustomMap.java:64)

at nl.basjes....@2.0-SNAPSHOT/nl.basjes.collections.MyCustomMap.put(MyCustomMap.java:31)

at com.esoteric...@5.0.0-RC9/com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:238)

at com.esoteric...@5.0.0-RC9/com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:42)

at com.esoteric...@5.0.0-RC9/com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:809)

at nl.basjes....@2.0-SNAPSHOT/nl.basjes.collections.MyCustomMapTest.testKyroSerialization(MyCustomMapTest.java:69)

Niels Basjes

[Niels Basjes]

I think I found why this is happening:

The list of default Serializers contains both 

the KryoSerializableSerializer ( https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/Kryo.java#L196 ) and 

the MapSerializer ( https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/Kryo.java#L215 )

in the order in which I expect them.

However this method effectively reverses the order of this initial set of serializers.

https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/Kryo.java#L361

To me an explicit serializer directive like the annotation and the KryoSerializable should always overrule any implicit serializer.

Niels Basjes

Op vrijdag 9 oktober 2020 om 12:08:26 UTC+2 schreef Niels Basjes:

[Thomas Heigl]

Hi Niels,

Thank you for looking into this issue!

You should be able to get your test working if you setup Kryo like this:

Kryo kryo = new Kryo();

kryo.register(MyCustomMap.class, KryoSerializableSerializer.class);

This forces Kryo to use the given serializer instead of falling back on the implicit default.

I'm not sure why such a low priority was chosen for `KryoSerializableSerializer`. It might make sense to move it to the end of the serializer chain.

Best regards,

Thomas

--
You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users
---
You received this message because you are subscribed to the Google Groups "kryo-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kryo-users/c984d799-d7d1-4355-bd37-bd53d4003c1dn%40googlegroups.com.

[Niels Basjes]

FYI:

I put up an attempt to fix this in Kryo:

https://github.com/EsotericSoftware/kryo/pull/772

Op vrijdag 9 oktober 2020 om 15:56:47 UTC+2 schreef thomas...@gmail.com: