Re: [protobuf] Attempting to build generic Message from serialized data regardless of type (c++)

[Ilia Mirkin]

Is it expected that the type of the message is available at
compile/run-time, or are you trying to load it dynamically? If
dynamically, you'd have to pass the descriptor as part of the message,
which is a little annoying (since it's not just the message, but also
any messages that that message references). If statically you should
be able to load the proto by name, you can get that out of
MessageFactory::generated_factory(), otherwise you have to use one of
the dynamic ones and load proto files/descriptor pb's you get over the
wire into it.

Usually with such generic situations, you might have a wrapper which would be

message TheData {
required string type = 1;
required bytes data = 2;
}

Or a more efficient arrangement like

<varint header length>
message Header {
required string type = 1;
anything else you want
}
<varint data length>
<the bytes of the proto>

This can be easily arranged for with a CodedInput/OutputStream in
Java, and I think there are similar constructs in the C++ impl.

An alternate technique, since you're using AMQP, is to attach meaning
to the channel key/name (or whatever that thing is called) and
determine the type from that.

On Wed, May 22, 2013 at 11:05 AM, Justin <jsca...@gmail.com> wrote:
> I'm receiving a serialized message over middleware (qpid), and I want to be
> able to instantiate a Message object in my base class which does not know
> the actual message type. I have a feeling this isn't actually possible, and
> I can easily workaround this by just passing the serialized message to the
> subclass instead of a Message object, but I would rather do it this way if
> possible.
>
> Here is the relevant code snippet:
>
> namespace GPB = google::protobuf;
>
> // Check the queue for a message
> qpid::messaging::Message message =
> amqpReceiver_.fetch(qpid::messaging::Duration::SECOND * 1);
> std::string msgContent = message.getContent();
>
> // Build a generic GPB message, without caring about the type.
> // Step 1: Create a message factory
> GPB::DynamicMessageFactory factory;
> // Step 2: Build a descriptor based on the message content
> GPB::Descriptor typeDescriptor(msgContent);
> // Step 3: Use the factory to make us an instance of the appropriate message
> type
> GPB::Message *gpbMessage = factory.GetPrototype(typeDescriptor).New();
>
> handleMessage(gpbMessage);
>
>
> This is all surrounded by a try/catch block, of course and I clean up memory
> after this.
>
> The handleMessage(Message *) method is a pure virtual that the subclasses
> implement depending on type. The qpid stuff is largely irrelevant, I just
> wanted to include mention of where it is coming from.
>
> My issue lies in the "Step 2", where I don't actually know how to make it
> work. The variable msgContent contains the serialized GPB message, but this
> class has no way of knowing the type. Is this even possible? I am pretty new
> to GPB, so I'm not sure if the serialized data even has enough information
> in it to allow generic deserialization and I'm partially assuming that it
> does not, but I'm hoping it does.
>
> I have read a bunch of stuff on forums, etc. regarding FileDescriptorProto
> and DescriptorPool, which may be part of the answer, but I'm not getting the
> whole picture yet.
>
> Thanks for any help you can offer!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+u...@googlegroups.com.
> To post to this group, send email to prot...@googlegroups.com.
> Visit this group at http://groups.google.com/group/protobuf?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Justin]

Thank you for your response Ilia. 

I was beginning to realize that I would have to provide some additional information for it to work beyond just the serialized data. I hadn't, however, thought of embedding it using the AMQP message options to make it work. Not sure why it didn't occur to me, but it's a good idea! thanks!