Deserialization of large message (> 64MB) slient-ly fails randomly.

[yangyi...@gmail.com]

Hi folks.

We are using c++ protocol buffer v3.0.2 for some structures' serialization and deserialization.

We are able to serialize a large PB message (~ 600MB) to a buffer and verfied the serialized size equals to PB.ByteSize(). 

Then when we try to deserialize the buffer back to the PB message. We are expecting the deserialization fails. But randomly, sometimes we see the deserialization succeeds, but the deserialized PB message only has a partial content with a ByteSize() == 64MB.

We wonder if there is anyone encountered this issue before. We would like to have a consistent behavior that all deserialization should fail (not succeed but return some wrong result).

serialization function we use: SerializeWithCachedSizesToArray(uint8 * target) const

deserialization function we use: ParseFromArray(const void * data, int size).

PB structure:

message response

{

    message A

    {

         required uint64 a = 1;

         repeated uint64 b = 2;

    }

    repeated A c = 1;

}

Thanks,

Yihao

[Adam Cozzette]

I would guess that there is a problem on the serialization side. Any of the WithCachedSizes serialization methods rely on the cached sizes, which means that you need to have called ByteSizeLong() or ByteSize() right before serializing. If the cached sizes are stale then things will most likely not work correctly.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/ed13812d-d115-4c10-a3a6-d07066698050n%40googlegroups.com.