Re: [protobuf] Merge serialized and non-serialized protobuf

[Bo Yang]

You can simply prepending the serialized tag and length before the serialized data of content.

Say, content is 0x0A0B0C0D, the you just prepend 0x12 (number is 2, type is length-delimited) and 0x04 (content is 4 bytes long).

So 0x12040A0B0C0D will be the data for wrapper.

On Fri, Jan 26, 2018 at 1:50 PM Mike Trienis <mtri...@quickinsights.io> wrote:

I'm trying to understand if it's possible to take a serialized protobuf that makes up part of another protobuf and merge them together without having to deserialize the first protobuf. 

For example, given a protobuf wrapper: 

syntax = "proto2";

import "content.proto";

message WrapperContent {
 // The schema version of the message envelope. For example, messageSchemaVersion = '1.1.1'
 required string metatData = 1;
 // The payload as well as any common fields across all events
 required Content content = 2;

}

And then imagine we get a serialized version of content below (i.e. content is coming that is coming from a remote client):

syntax = "proto2";

message Content {
 // The schema version of the message envelope. For example, messageSchemaVersion = '1.1.1'
 required string name = 1;
 // The payload as well as any common fields across all events
 required bytes payload = 2;

}

Do you know if any way I can inject the serialized Content into the WrapperContent without first having to deserialize Content. 

The reason I'm trying to inject Content without deserializing it, is to try and save on the overhead of deserializing the message. 

Thanks, Mike.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Adam Cozzette]

Another thing you could do is create another message which exactly matches WrapperContent except that it stores a bytes field instead of the Content field, like this:

message WrapperContentBytes {

  required string metatData = 1;

  required bytes content = 2;

}

Then you can just stick the serialized Content proto directly in the content field without parsing it, and when you serialize the WrapperContentBytes proto you will end up with a string that should be parsable as a valid WrapperContent proto. The reason this works is that the wire format represents an embedded message the same way it represents a string or bytes field.

One thing to watch out for is that if the serialized Content proto is somehow malformed, you won't find out about that until later on when you try to parse the WrapperContent containing it.

On Fri, Jan 26, 2018 at 1:49 PM, Mike Trienis <mtri...@quickinsights.io> wrote:

I'm trying to understand if it's possible to take a serialized protobuf that makes up part of another protobuf and merge them together without having to deserialize the first protobuf. 

For example, given a protobuf wrapper: 

syntax = "proto2";

import "content.proto";

message WrapperContent {
 // The schema version of the message envelope. For example, messageSchemaVersion = '1.1.1'
 required string metatData = 1;
 // The payload as well as any common fields across all events
 required Content content = 2;

}

And then imagine we get a serialized version of content below (i.e. content is coming that is coming from a remote client):

syntax = "proto2";

message Content {
 // The schema version of the message envelope. For example, messageSchemaVersion = '1.1.1'
 required string name = 1;
 // The payload as well as any common fields across all events
 required bytes payload = 2;

}

Do you know if any way I can inject the serialized Content into the WrapperContent without first having to deserialize Content. 

The reason I'm trying to inject Content without deserializing it, is to try and save on the overhead of deserializing the message. 

Thanks, Mike.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.