Why unsafe-based IO can have singficant performance boost

[Jun Li]

I am using Spark, and to improve object serialization/de-serialization performance, I always choose kryo serializer. Then I realized that on the Kryo github site, there is a section on "Unsafe IO", and it states:

Using Unsafe-based IO may result in a quite significant performance boost (sometimes up-to an order of magnitude), depending on your application. In particular, it helps a lot when serializing large primitive arrays as part of your object graphs.

could someone comment on why use Unsafe-basedIO stream can perform much better than  through for example,  the stream that is based on ByteBufferInputStream/ByteBufferOutputStream?


Jun

[mongonix]

Unsafe-based IO uses methods from sun.misc.Unsafe for reading and writing from/to memory. Many of those methods map almost 1:1 to processor instructions. Moreover, reading/writing of arrays of native types is executed in bulk instead of doing it element-by-element as it is usually done by Kryo. Together, these features often provide a significant performance boost. 

-Leo

 

Jun