how to calculate protocol buffer message size without parsing the full message?

5,784 views
Skip to first unread message

Sean Nguyen

unread,
Apr 21, 2013, 12:07:57 AM4/21/13
to prot...@googlegroups.com
Hi,

I want to calculate the protocol buffer message size without reading the stream to the end? I don't see the size of the outer most message is stored. For example:

message Person {
   optional string name = 1;
   optional uint32 age = 2;
}

There is no size store for the Person message if it is the outer most message. Is that correct? If it is not where I can find it?

Thanks,

Son Nguyen

Marc Gravell

unread,
Apr 21, 2013, 3:00:27 AM4/21/13
to Sean Nguyen, prot...@googlegroups.com
The outermost message is not stored with a length; this allows multiple messages to be merged by concatenation. The default behaviour for a protobuf parser at the root-object is "read until you run out of data". If you have written multiple messages without some kind of "framing" (usually adding length-prefixes manually) then it will be very hard to unscramble the data. If you need to append individual messages and want to parse them back individually, one option is to encapsulate each in a parent object with a "repeated" element, with a single element per object. Alternatively, add your own frames.

Marc
--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Oliver Jowett

unread,
Apr 22, 2013, 4:16:00 AM4/22/13
to Oliver Wang, Protocol Buffers
On Sun, Apr 21, 2013 at 9:48 PM, Oliver Wang <pyth...@gmail.com> wrote:
Can't you use ByteSize() method to get the size?

Not when you're parsing - you don't have a message instance yet!

Oliver

Reply all
Reply to author
Forward
0 new messages