Can we Compress and decompress

[nit]

HI All,
Can anyone let me know whether the encoded file can be compressed
while sending it and decompressed at the receiver's end?
The problem is like i have a message structure which has many field
and consumes more bytes. So can i compress and Decompress them?

thanks,
Nithin Shubhananda

[Marc Gravell]

Explicit compression isn't built into protobuf, since it is very dense to start with. In some cases it may be successful to run your data through GZIP or DEFLATE - in particular if your data has a lot of repeated data in strings, etc. In other scenarios I've seen GZIP / DEFLATE make protobuf data *bigger*. So:

a: you would have to do it manually

b: whether it will help depends on your data

Marc

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

--
Regards,

Marc

[Kenton Varda]

Compression is not a built-in feature of protocol buffers, but it is easy and often useful to apply compression on top of the protobuf encoding.

In Java, write your data to a GZIPOoutputStream, and then read from a GZIPInputStream.  These classes are part of the standard Java library, not the protobuf library.

In C++, use the GzipOutputStream and GzipInputStream classes that are provided in the protobuf library (under google/protobuf/io).  These simply call zlib to apply compression on top of protocol buffers.

[Igor Gatis]

Are C++ and Java GZip streams compatible?

[Marc Gravell]

In terms of output? GZIP / DEFLATE / etc implementations should be interoperable between platforms, yes. However IIRC the compressed stream is not guaranteed to be byte-identical between specific implementations (but the *decompressed* stream should be byte-identical)

Marc

Regards,

Marc

[Igor Gatis]

I mean: if I use Java's GzipOutputStream and then read compressed data using C++'s GzipInputStream, will it work?

In other words, are generated streams pure Gzip compliant, in the sense there are no special headers. Could I use unix gzip to unzip a file which its content was generated by Java or C++ GzipOutputStream?

Sounds like the answer is yes. Just wanted to confirm that.

[Kenton Varda]

On Tue, Oct 19, 2010 at 1:10 PM, Igor Gatis <igor...@gmail.com> wrote:

In other words, are generated streams pure Gzip compliant, in the sense there are no special headers. Could I use unix gzip to unzip a file which its content was generated by Java or C++ GzipOutputStream?

Yes.  For C++, the protobuf code actually includes a test verifying this.  The Java class is not our code but I would assume it works the same.