Python fails to parse protobuf structure in http body

[Irad K]

Hi, I wonder if anybody can help me out with a Protobuf parsing issue I'm currently facing. So I've implemented the following protobuf based protocol

message singleConfig { 

  string configName = 1;

  string configValue = 2; 

} 

message currentConfig { 

  repeated singleConfig conf = 1; 

} 

message HttpRequest {

  string osVersion = 1;

  string productVersion = 2;

 currentConfig config = 3; 

}

I send this object HttpRequest, on the body of post http request from the c++ based client using the following serialization: 

protocol::ProtocolRequest &requestBody;

// fill up the protobuf object  

req.body() = requestBody.SerializeAsString();

On my http python server, I expect to get http post requests from body that conform this protocol.

So upon incoming http post request, the body contents arrived and seems valid (I can identify the fields' values from the text)

b'2@\n\x0611.5.1\x12\x061.0(1)\x1a.\n,\n\x08file.json\x12 ecf1c21c77a419f8f7dbfb714a806166'

Here's the code that parse the http request. notice that ParseFromString accept 'bytes' formatted input. The parsing finish without any exception so I assume it went alright...

message = HttpRequest() 

message.ParseFromString(data)

However, an attempt to access each one of the fields in the protobuf structure reveals empty value :

message.osVersion 

''

message.productVersion 

''

...

Any idea what's wrong with the parsing, is it the fact that I'm using python3? bad formatting ? should I pass the body not as a string but from different encoding ?

Thanks !

[Adam Cozzette]

I think something is wrong with that serialized proto. There may be a valid proto in there somewhere, but at least the first few bytes seem to be wrong. I starts with 0x32 which would indicate field number 6 and wire type 2 (see encoding details here), but your schema does not have a field number 6 anywhere. I would guess that some extra bytes are getting prepended to the serialized proto.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/760e7e07-15c5-42db-84ef-a20564f5481bn%40googlegroups.com.

[Irad K]

Yes, you where right about the first 2 bytes, it was indeed what's corrupted my Protobuf structure. Thanks !

ipdb> message.ParseFromString(data[2:])

64

ipdb> message.osVersion

'11.5.1'

[Irad K]

I believe the extra 2 bytes where related to the fact that HttpRequest Protobuf struct is in fact part union object that may include several other Protobuf structure, and those bytes where identifier that the selected struct is in fact HttpRequest.

Do you know how do parse the correct struct from the union (oneof) from the raw data? any method I should look for in the union struct ?

[Adam Cozzette]

You can just parse the containing message normally and then use the usual accessors to access the elements inside the oneof.

To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/c7c06ec5-c28d-482e-bec2-9e24fed4bea9n%40googlegroups.com.