std::system_error occurs at Protobuf ParseFromZeroCopyStream()

[HyonChang Lim]

I'm testing protobuf with zlib compression.

I wrote some c++ sample code using protobuf 3.8.0, but the following error occurred at calling ParseFromZeroCopyStream() at Ubuntu.

terminate called after throwing an instance of 'std::system_error'
  what():  Unknown error -1
(core dumped)

what can I do?

I tried to replace ParseFromZeroCopyStream() with ParseFromBoundedZeroCopyStream(). That results in no core dump, but ParseFromBoundedZeroCopyStream() returned false.

test.proto

syntax = "proto2";

package test;

message Msg
{
    required uint32 data = 1;
}

test.cc

#include <iostream>
#include <google/protobuf/io/zero_copy_stream_impl.h>
#include <google/protobuf/io/zero_copy_stream_impl_lite.h>
#include <google/protobuf/io/gzip_stream.h>
#include "test.pb.h"

using namespace std;
using namespace google::protobuf;
using namespace test;

int main(void)
{
    Msg srcMsg; 
    srcMsg.set_data(1);

    long sSize = srcMsg.ByteSizeLong();
    cout << "SerializedSize = " << sSize << endl;

    char * compressedMsg = new char[sSize];
    io::ArrayOutputStream aos(compressedMsg, sSize); 
    io::GzipOutputStream gos(&aos);

    long cSize;
    if (srcMsg.SerializeToZeroCopyStream(&gos) == true)
    {
        gos.Close();
        cSize = aos.ByteCount();
        cout << "compression success : " << cSize << " bytes" << endl;
    }
    else    
    {
        cout << "compression error" << endl;
        return 1;
    }

    Msg targetMsg;

    io::ArrayInputStream ais(compressedMsg, cSize); 
    io::GzipInputStream gis(&ais);

    if (targetMsg.ParseFromZeroCopyStream(&gis) == false)
    {
        cout << "decompression error" << endl;
    }
    else    
    {
        cout << "decompression success : " << targetMsg.ByteSizeLong() << " bytes" << endl;
        cout << "data = " << targetMsg.data() << endl;
    }

    delete[] compressedMsg;

    return 0;
}

I expect that decompression succeeds.

[Adam Cozzette]

I suspect the problem is that the array you're creating for the serialized data is too small. That message's serialized size is only 2 bytes, so gzipping it will likely increase the size, and the gzipped data will not fit in a 2-byte array. I would recommend serializing to a std::string using a StringOutputStream, since that will automatically resize the string.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/8b697653-58dd-46c5-b442-aa4b76388c04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[HyonChang Lim]

Hi, Adam

Your answer is very helpful. Thanks.

1. I tried increase the size of array like that.

        char * compressedMsg = new char[1000];

        io::ArrayOutputStream aos(compressedMsg, 1000);

   works.

2. Instead of using ArrayOutputStream, StringOutputStream works also.

Thanks again.

2019년 7월 9일 화요일 오전 7시 0분 55초 UTC+9, Adam Cozzette 님의 말:

I suspect the problem is that the array you're creating for the serialized data is too small. That message's serialized size is only 2 bytes, so gzipping it will likely increase the size, and the gzipped data will not fit in a 2-byte array. I would recommend serializing to a std::string using a StringOutputStream, since that will automatically resize the string.

To unsubscribe from this group and stop receiving emails from it, send an email to prot...@googlegroups.com.