Decode callback not being called for oneof entry

[Andrew Pullin]

I am having an issue with nanopb, unsure if it is a bug or just my incorrect usage.

I have a oneof message, and at the decode step, the decode callback function is not getting called at all for the message inside the oneof, but it is getting called for the same message structure that is present again outside of the oneof statement.

I am using an example proto file setup like this (simple example to reproduce this issue):

syntax = "proto2";

import 'nanopb.proto';

message TestMessage {

    message TestType1 {

        required int32 test_int = 1;

        required string test_str = 2;

    }

    message TestType2 {

        required string test_str = 3;

    }

    oneof message {

        TestType1 test_type_1                      = 4;

        TestType2 test_type_2                      = 5;

    };  

    required TestType1 test_outside = 6;

}

I am setting up the decode callbacks with:

    proto_msg.message.test_type_1.test_str.funcs.decode = print_string;

    proto_msg.message.test_type_1.test_str.arg = "stringvalue: \"%s\"\n";

    proto_msg.message.test_type_2.test_str.funcs.decode = print_string;

    proto_msg.message.test_type_2.test_str.arg = "stringvalue: \"%s\"\n";

    proto_msg.test_outside.test_str.funcs.decode = print_string;

    proto_msg.test_outside.test_str.arg = "stringvalue: \"%s\"\n";

Any my decode step is getting called with:

    status = pb_decode(&stream, TestMessage_fields, &proto_msg);

But only the string in test_outside.test_str is getting printed, and print_string is only called once.

However, I know the data is encoded correctly, as the same binary decodes correctly with the nanopb-generated python module.

Is this a usage error or a bug?

A complete test program (which also does bin-to-hex and hex-to-bin conversion, because my real application uses it, but also this lets me copy&paste into python test) is attached, along with the proto file.

[Petteri Aimonen]

Hi,

> I have a oneof message, and at the decode step, the decode callback
> function is not getting called at all for the message inside the oneof, but
> it is getting called for the same message structure that is present again
> outside of the oneof statement.

Yeah, it is a known issue:
https://github.com/nanopb/nanopb/issues/175

Callbacks inside oneofs are currently not supported. Problem is that if
the oneof is stored as C union, the callbacks between different fields
would occupy same memory area.

Workarounds include specifying (nanopb_msgopt).no_unions = true, or
using a callback for the oneof field also.

--
Petteri

[Andrew Pullin]

Aha. 

 

using a callback for the oneof field also.

I am not quite sure what this might look like.... I guess there is a way to force nanopb to generate a callback for the whole oneof struct, and then I could do processing inside that callback?

[Petteri Aimonen]

Hi,

> I am not quite sure what this might look like.... I guess there is a way to
> force nanopb to generate a callback for the whole oneof struct, and then I
> could do processing inside that callback?

Yeah, something like that. Though I guess there isn't a clean way to
have a combined callback for all fields in oneof, so it would still boil
down to having a separate callback for each of the fields inside the
oneof.

It's an annoying problem and gets asked about often, but I haven't
figured out a good solution. I feel like the callbacks should be
taken entirely out of the message structure, and be given separately to
the decoding function somehow.

--
Petteri

[Andrew Pullin]

Interesting. It looks like I'll just have to switch to the non-union type for now.

Does nanopb presently do any error checking to verify that the oneof is respected at encode/decode time, such that only one has_xyz is allowed?

Maybe this could be accomplished by allowing a "decode setup" callback to be executed as soon as the tag type for the oneof is decoded, which would allow the correct decoder functions to be installed before the decode for the target struct continues?
But nanopb is intricate enough that I wouldn't try to implement this myself ;)

Thanks,
Andrew

[Petteri Aimonen]

Hi,

> Does nanopb presently do any error checking to verify that the oneof is
> respected at encode/decode time, such that only one has_xyz is allowed?

No, in fact it is legal in protobuf format to have the same oneof
multiple times in same message. Later instances will overwrite the
earlier ones.

> Maybe this could be accomplished by allowing a "decode setup" callback to
> be executed as soon as the tag type for the oneof is decoded, which would
> allow the correct decoder functions to be installed before the decode for
> the target struct continues?

Yeah, this is one of the more promising approaches. I haven't had time
to implement it (or figure out the details), though.

--
Petteri

[Girish SG]

Is this issue resolved now with nanopb-0.4.7-macosx-x86.tar.gz version or it still exists?