BufferUnderflowException

[Hsaka]

Hey Nate, i got this exception a few minutes ago. The server app
crashed and threw the following exception:

Exception in thread "Server"
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:
474)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:
117)
at
com.esotericsoftware.kryo.serialize.ShortSerializer.get(ShortSerializer.java:
71)
at
com.esotericsoftware.kryo.serialize.ShortSerializer.readObjectData(ShortSerializer.java:
29)
at
com.esotericsoftware.kryo.serialize.ShortSerializer.readObjectData(ShortSerializer.java:
14)
at
com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:
56)
at
com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:
176)
at
com.esotericsoftware.kryo.Compressor.readObjectData(Compressor.java:
95)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:
307)
at
com.esotericsoftware.kryonet.TcpConnection.readObject(TcpConnection.java:
121)
at com.esotericsoftware.kryonet.Server.update(Server.java:
178)
at com.esotericsoftware.kryonet.Server.run(Server.java:
319)
at java.lang.Thread.run(Thread.java:619)

Using kryo-0.92 and kryonet-0.91
Around 60 people were online.
What could be the cause?

[Nate]

Hi Hsaka,

Hmmm. KryoNet read a length from a client, read that many bytes, tried to deserialize those bytes, during which it ran out of bytes. The reason was that the length the client sent for the object was too short. Possible causes:

1) A malicious client sending bad data.

2) TCP uses a 16 bit checksum. It is possible for multiple errors to occur to a packet in such a way that the checksum is not invalidated. However, this is quite rare.

3) Reads are expected to happen on the network thread, so are not synchronized like writes. If you are calling Server#update from multiple threads, the state of the read buffer would be corrupted. If you use Server#start rather than calling update yourself, and you call start only once, then this isn't the issue.

4) A serializer turns an object into X bytes, but then tries to read more than X bytes during deserialization.

Assuming only one thread is calling update, it isn't possible that a previous object that was read affected the object that failed, because KryoNet checks to make sure that the correct number of bytes were used after deserialization. So if a client sends a length that is too long or a serializer otherwise doesn't use all the bytes, it blows up right then rather than corrupting the next read.

From your stacktrace, I can see you are using a compressor and that you have set it to not decompress. I assume you have the client correctly set up to not compress?

You say you had 60 users online, that is fantastic! :) The popularity, not the crash. ;) Your app probably has more users than other KryoNet apps so far. I've got one in the works but I've been busy lately.

So of the above possibilities I could think of, #4 seems the most likely. With so many users, I'm guessing this doesn't happen very often? If so and if the problem is #4, this leads me to think that some piece of data isn't being serialized or deserialized correctly.

Ah, I should have noticed how old your library versions are! There have been a few changes to KryoNet since 0.92, but a *lot* of changes to Kryo since 0.91. Looking at the change history, it looks like Kryo revision 60 could be the culprit, where StringSerializer could read too many bytes. In revision 68 the StringSerializer was completely replaced with a much faster implementation. Kryo now kicks ass in the benchmarks, approaching hand written serialization:
http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking

Could you update to the latest? I just did new releases to both projects. The latest is v1.0! :)

All that said, the server should not crash when one client sends bad data. I'll work on a fix for this soon.

-Nate

--
You received this message because you are subscribed to the "kryonet-users" group.
http://groups.google.com/group/kryonet-users

[Nate]

In Kryo, I've made both buffer underflow and buffer overflow throw SerializationException. In KryoNet, when serialization fails with this exception, the problematic connection will be closed. The entire server will no longer crash.

This is in SVN (be sure to get latest from both projects) and will be in versions > 1.0.

-Nate

[Hsaka]

Awesome Nate, thanks alot.
I'll update to the latest version tonight.

[Hsaka]

Hi Nate,

I'm using the latest version of kryo and kryonet from svn. My
application is a Pulpcore app and needs to be signed. When trying to
sign it, however, I get the following:

jarsigner: unable to sign jar: java.util.zip.ZipException: duplicate
entry: org/objectweb/asm/AnnotationVisitor.class

Not sure what to do here.

[Nate]

Hi Akash,

Seems to be a common issue on The Google. You might check this:
http://www.j2meforums.com/forum/index.php?topic=12114.0

-Nate

To unsubscribe from this group, send email to kryonet-users+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.

[Hsaka]

Hey Nate,

I tried a few things to get the jar to be signed properly, but none of
them worked. So, I hacked up the latest kryo a bit to not use
asm-3.2.jar, which I saw only FieldSerializer uses. I removed the
entire AccessLoader class.

The app is now signed properly, but what effect will changing
FieldSerializer in this way have? All the other classes are from the
latest kryo in svn.

[Nate]

FieldSerializer uses ASM to generate bytecode at runtime to get and set public fields. This is faster than using reflection, as seen here:
http://code.google.com/p/reflectasm/
Note that if your fields are not public then the ASM magic can't be used and reflection will be used instead.

I am able to individually sign the KryoNet JAR and its dependency JARs. I see for All Fours Online that you combine all your JARs into one. I did this will all the KryoNet JARs and was still able to sign it.

Maybe you could send me your unsigned JAR so I can try to sign it to see the failure first hand? I'm guessing the MANIFEST.MF file has bogus entries that are messing up the sign. You might try deleting any "Name:" and "SHA1-Digest:" entries before attempting to sign.

-Nate

To unsubscribe from this group, send email to kryonet-users+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.