Forced serialization/deserialization of unknown fields in proto3 messages

[Jeremy Ong]

Hi google pb,

I was wondering if an interface exists for specifying that I do not want the proto3 serialization or deserialization to discard unknown fields. My understanding was that this change was made from proto2 to proto3, and is a pretty severe restriction if there are no ways around it. The motivating example in my case is to potentially decorate a message with fields in the extension ranges that are not part of the message body. The meaning is purely semantic and I do not want the data therein to be contained in the protobuf format itself. If unknown fields are not an option, are there other options or suggestions to handle this?

Best,

Jeremy

[Feng Xiao]

On Tuesday, March 15, 2016 at 12:32:07 PM UTC-7, Jeremy Ong wrote:

Hi google pb,

I was wondering if an interface exists for specifying that I do not want the proto3 serialization or deserialization to discard unknown fields. My understanding was that this change was made from proto2 to proto3, and is a pretty severe restriction if there are no ways around it. The motivating example in my case is to potentially decorate a message with fields in the extension ranges that are not part of the message body. The meaning is purely semantic and I do not want the data therein to be contained in the protobuf format itself. If unknown fields are not an option, are there other options or suggestions to handle this?

Some alternatives to consider:

1. use a bytes field to store these data, and decode it manually if it's a proto.

2. use an google.protobuf.Any field for it if the data is still a protobuf message.

 

Best,

Jeremy

[Jeremy Ong]

Neither are appropriate in my use case unfortunately. I want to be
able to tag any message with data in a field range special within the
organization. The point is that I don't want to add fields to the
existing hundreds and hundreds of message types we have already.

For the time being, I have switched everything back to proto2, which
despite some inconvenience I believe is more feature complete and
superior to proto3 at this time.

> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+u...@googlegroups.com.
> To post to this group, send email to prot...@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.

[Steven Parkes]

You can do something like this by serializing the normal message and the added fields separately and then concatenating the byte strings.

Not helpful if you need to do this above the byte string level. And may have other disadvantages, e.g., extra copies ...

[Jeremy Ong]

Thanks for the suggestion but this would mean that every message would need to be packed in this structure including nested messages. It's simply too much for a hack for what I believe was a simple and useful feature that was part of proto2 and removed with no explanation or way to disable it.

Jeremy Ong

PlexChat CTO

301.648.8260