Protobuf as generic marshalling layer

[Galder Zamarreno]

Hi,

I want to use protobuf as a generic marshalling mechanism and I'm
having some issues with the unmarshalling part, because I need
knowledge of what needs to be unmarshalled, i.e. in Java, access to
static parseFrom method in target class, which makes it hard to use it
in a generic way. A hacky workaround for the java world would be to
write the class name as UTF-8 string and after instantiating it in the
reading part, use reflection. Obvioulsy, this would only work for the
java world. Any ideas of similar solutions for the Python/C++ world?

I've also read about self-describing messages but that API is only
available for C++ and Java and not Python, so it's not enough.

Cheers,

[Jason Hsueh]

On Tue, Jul 13, 2010 at 1:38 AM, Galder Zamarreno <mark...@zamarreno.com> wrote:

Hi,

I want to use protobuf as a generic marshalling mechanism and I'm
having some issues with the unmarshalling part, because I need
knowledge of what needs to be unmarshalled, i.e. in Java, access to
static parseFrom method in target class, which makes it hard to use it
in a generic way. A hacky workaround for the java world would be to
write the class name as UTF-8 string and after instantiating it in the
reading part, use reflection. Obvioulsy, this would only work for the
java world. Any ideas of similar solutions for the Python/C++ world?

Assuming that you have the full descriptors available on both ends, you can take the approach of sending the type name. In C++ you can also use reflection by looking up the Descriptor objects by type name (see the DescriptorPool class) and creating DynamicMessages out of that, much like you would in Java.

But I think you are correct that there's no API for Python to look up descriptors by typename.

I've also read about self-describing messages but that API is only
available for C++ and Java and not Python, so it's not enough. 

In theory it should be possible to do this in Python as well using reflection.py. You should be able to take serialized FileDescriptorProtos and create FileDescriptor objects from them. I'm not yet sure of the details - I will investigate and let you know what I find (or hopefully someone with more familiarity with the Python API will chime in :-) )

Cheers,

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[Kenton Varda]

If you are writing a library, and the client of your library knows the exact type desired, then they should pass to you the class's default instance (C++ and Java; in Python they might as well pass the class itself).  In C++ this is MyType::default_instance(), in Java MyType.getDefaultInstance().  Given the default instance, you can call various methods on the interface to do everything you need.  E.g. to parse:

C++:

  Message* instance = default_instance->New();

  instance->ParseFromString(data);

Java:

  Message.Builder builder = defaultInstance.newBuilderForType().mergeFrom(data);

  if (!builder.isInitialized()) {

    return false;

  }

  Message instance = builder.build();

On Tue, Jul 13, 2010 at 1:38 AM, Galder Zamarreno <mark...@zamarreno.com> wrote: