kryo-serializers/SynchronizedCollectionsSerializer: Buffer underflow.

[Martin Grotzke]

Hi,

the test for kryo-serializers' SynchronizedCollectionsSerializer fails
now with kryo2 with this exception:

com.esotericsoftware.kryo.KryoException: Buffer underflow.
at com.esotericsoftware.kryo.io.Input.require(Input.java:140)
at com.esotericsoftware.kryo.io.Input.readInt(Input.java:276)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:84)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:1)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:643)
at
de.javakaffee.kryoserializers.SynchronizedCollectionsSerializer.read(SynchronizedCollectionsSerializer.java:56)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:537)
at de.javakaffee.kryoserializers.KryoTest.deserialize(KryoTest.java:632)
at
de.javakaffee.kryoserializers.KryoTest.testJavaUtilCollectionsSynchronizedList(KryoTest.java:259)

The working draft for kryo2 is in kryo-serializers kryo2 branch on
github btw.

This is the test case:
https://github.com/magro/kryo-serializers/blob/kryo2/src/test/java/de/javakaffee/kryoserializers/KryoTest.java#L257

The methods serialize/deserialize look like this:

protected byte[] serialize( final Object o ) {
if ( o == null ) {
throw new NullPointerException( "Can't serialize null" );
}

final ByteArrayOutputStream outStream = new ByteArrayOutputStream();
final Output output = new Output(outStream, 4096);
_kryo.writeObject(output, o);
output.flush();
return outStream.toByteArray();
}

protected <T> T deserialize( final byte[] in, final Class<T> clazz ) {
final Input input = new Input(new ByteArrayInputStream(in), in.length);
return _kryo.readObject(input, clazz);
}

(just new Input(new ByteArrayInputStream(in)); doesn't change anything btw)

Here you see the SynchronizedCollectionsSerializer:
https://github.com/magro/kryo-serializers/blob/kryo2/src/main/java/de/javakaffee/kryoserializers/SynchronizedCollectionsSerializer.java

I'm still at revision 186 btw.

Cheers,
Martin

[Sam Ritchie]

I was getting the same error when wrapping an OutputStream directly with output in cascading.kryo.

--

Sam Ritchie, Twitter Inc

703.662.1337

@sritchie09

(Too brief? Here's why! http://emailcharter.org)

[Martin Grotzke]

I just debugged this a bit and found that in
testJavaUtilCollectionsSynchronizedList

1. Kryo.writeReference is invoked with the
Collections.SynchronizedRandomAccessList and stores 0 for the
synchronized list
2. Kryo.writeClassAndObject is invoked with the wrapped ArrayList from
SynchronizedCollectionsSerializer
3. Kryo.writeClassAndObject invokes writeReference with the wrapped list
4. which gets the 0 from objectToInstanceId and only writes the int
reference to the output.

To me this seems to be the issue: the objectToInstanceId Map does not
use the identityHashCode but the normal hashCode.
When I change the ObjectIntMap to use System.identityHashCode in get and
put for the key the issue is solved and the test is green.

Cheers,
Martin

On 04/12/2012 09:09 PM, Sam Ritchie wrote:
> I was getting the same error when wrapping an OutputStream directly with
> output in cascading.kryo.
>
> On Thu, Apr 12, 2012 at 11:40 AM, Martin Grotzke

> <martin....@googlemail.com <mailto:martin....@googlemail.com>>

> --
> You received this message because you are subscribed to the "kryo-users"
> group.
> http://groups.google.com/group/kryo-users

--
Brakhane, Grotzke und Langbehn Informatiker und Physiker PartG
Breitenfelder Str. 13c, 20251 Hamburg
Amtsgericht Hamburg, PR 795

[Nate]

Thanks for debugging this. Fixed in SVN!

-Nate

[Nate]

Note if your input is byte[], there is no need to wrap it in an InputStream.

protected <T> T deserialize( final byte[] in, final Class<T> clazz ) {

   final Input input = new Input(in);
   return _kryo.readObject(input, clazz);
}

-Nate

[Martin Grotzke]

I already noticed this, thanx for the hint!