Varints and wire-format

39 views
Skip to first unread message

Lukasz

unread,
May 8, 2019, 5:02:47 AM5/8/19
to Protocol Buffers
Hello,

The current definition of varint on the wire is following (https://developers.google.com/protocol-buffers/docs/encoding#varints):

Each byte in a varint, except the last byte, has the most significant bit (msb) set – this indicates that there are further bytes to come. The lower 7 bits of each byte are used to store the two's complement representation of the number in groups of 7 bits, least significant group first.

I wonder if i can strictly depend on this definition and make from varint something like fixed-length-varints, for example i would like to encode "0" or "1" as varint using 4 bytes.

Example of "0" encoding:
LSbyte...           MSbyte
0x80 0x80 0x80 0x00

Example of "1" encoding:
LSbyte...           MSbyte
0x81 0x80 0x80 0x00

I know that it might be something that "breaks" the key-idea behind the varints, still i am doing this to keep my serializer simple and performant. My question is can i depend on such behavior or its something that might be changed in future by adding some "new magic" to this varint-wire-format ?

Best Regards,
Lukasz

Adam Cozzette

unread,
May 8, 2019, 11:20:57 AM5/8/19
to Lukasz, Protocol Buffers
Yes, that should work. Now I forget what it is called but I came across an open source C++ protobuf library that implemented this exact idea and used 4-byte varints for all length prefixes. It would be interesting to see if this turns up bugs in any parsers, but in principle any correct parser should be able to handle a varint that uses more bytes than strictly necessary.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/f7fd31e7-e439-43a3-b7f0-52a1108917ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages