Persisting protobufs to disk

30 views
Skip to first unread message

Matthew Eck

unread,
Mar 9, 2020, 6:30:10 PM3/9/20
to Protocol Buffers
Is there a recommended, documented format for reading and writing protobuf mesages to/from a file? 
Preferably without converting to JSON. I seek something equivalent to Google's internal RecordIO format, which AFAICT is not published; something like Avro object container file, except for protobuf. 
Thanks.

Adam Cozzette

unread,
Mar 9, 2020, 7:18:50 PM3/9/20
to Matthew Eck, Protocol Buffers
There's not really an official recommended format for doing this. Probably the easiest thing is to store the raw serialized message, if you can fit everything into one message in a reasonable way without it being too big. The other common approach is just to store a simple series of messages, with each message prefixed by its size as a varint. This is sometimes called the "delimited" format and for some languages we include support for it. For example here is the code implementing it in C++: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/util/delimited_message_util.h

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/2ba87854-c284-4b42-adef-67f8e4c41081%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages