dynamic generation of proto objects and the error: Assignment not allowed to composite field

[Tom Lichtenberg]

I’m trying to dynamically create objects from discovery of pb2 generated protobuf files and assign appropriate values to their fields. If the field types are simple, this is straightforward - e.g. for TYPE_BOOL, setattr(obj, “field_name”, True) works fine - but if the field is TYPE_MESSAGE, making an instance of that field and using setattr(obj, “field_name”, instance) results in an error:

      Assignment not allowed to composite field "task" in protocol message object.

from python2.7/site-packages/google/protobuf/internal/python_message.py", line 736, in setter

sample code: dummy/test_dummy.proto

============================

syntax = "proto3";

message task {

   int32 id = 1;

   string msg = 2;

}

message task_info {

   task task = 1;

}

from this I used protoc to generate a test_dummy_pb2.py

For example, I want to create a task_info object, so by exploring the imported pb2 module, I find the task_info descriptor and create an instance of it. I see that its field ‘task’ is of TYPE_MESSAGE so I create an instance of that as well and call setattr(task_instance, “id”, 1) and setattr(task_instance, “msg”, “some message”) to populate it, and then go back and call setattr(task_info_instance, “task”, task_instance). Here I get the error:  Assignment not allowed to composite field "task" in protocol message object.

programatically I can import this module and assign values well enough, for example in the python shell:

>> import importlib

>> oobj = importlib.import_module("dummy.dummy.test_dummy_pb2", package=None)

>> obj = getattr(oobj, "task_info")

>> task_info_instance = obj()

>> task_info_instance.task.id = 1

>> task_info_instance.task.msg = "some message"

>> print task_info_instance

task {

  id: 1

  msg: "some message"

}

  

But dynamically I can’t explicitly assign it like that. I would need something like a setattr() function that can set an object’s composite object field. I’m hoping that someone can shed some light on this for me.

[Tom Lichtenberg]

One other thing I am now trying is to take the task_instance object and call SerializeToString() on it, then use task_info_instance.MergeFromString(serialized_task_instance)

this is not raising any exception and does seem to be setting the field, but I'm not entirely convinced yet

[Tom Lichtenberg]

The real solution seems to be to build the objects from the bottom up. What's going on is that nested messages are "immmutable" objects from the point of view of their parent object. You can't just set them from the top down, but you can build the object from the bottom up. The error messages and protobug docs don't use the words 'immutable' or 'mutable' but their use would explain a lot (at least to me)

For example:

syntax = "proto3";

message SubTask {
   string secret = 2;
}

message Task {

   int32 id = 1;
   string msg = 2;

   SubTask mission = 3;
}

message TaskInfo {
   Task task = 1;
}

if we want to create a TaskInfo object dynamically, we first need to create a SubTask object, then create a Task object passing in the SubTask object in its constructor, then create the TaskInfo object passing in the Task object in that constructor

args = { "secret": "secret message"}

subtask = SubTask(**args)

args = { "id": 100, "msg", "some message", "mission": subtask }

task = Task(**args)

args= {"task": task}

task_info = TaskInfo(**args)

In this way you can also use variables for the field names in the **args dictionary when creating objects dynamically for any random proto, where you don't know the field names apriori but discover them. Create the sub_messages first, then their parent, then its parent and so on, from the bottom up

[Tom Lichtenberg]

The original intention was to be able to do this with dynamically discovered messages for which you have the as set of *_pb2.py objects but don't know in advance which one you'll need to instantiate and set the fields for

assuming your **args contains keys and values for all of the required fields in the protobuf object, you can use importlib to load the protobuf message from the module, and use setattr to set each key-value pair

obj = getattr(importlib.import_module(class_name), message_name)
for descriptor in obj.DESCRIPTOR.fields:
    print descriptor.name
    setattr(obj, descriptor.name, args[descriptor.name])

if the message contains fields that are themselves messages (type=TYPE_MESSAGE) you need to instantiate the embedded message object first and assign that created object to its corresponding key in the args dict

you'd need to do this for embedded messages all the way down - so you need to fully parse the object, discover all the embedded messages, and build your object from the bottom up

On Friday, March 3, 2017 at 1:09:09 PM UTC-8, Tom Lichtenberg wrote:

[Anton Danilov]

if it's still interested for you AFAIK syntax = 2 required for dynamic protobufs