How to dynamically parse protobuf with extension messages

4,131 views
Skip to first unread message

prem

unread,
Oct 11, 2011, 12:43:32 PM10/11/11
to Protocol Buffers
On one machine, serialized a Message2, which has been extended from
Message1.

The parameters in Mesage2 were set using setExtension API. Did
writeDelimitedTo to write to a byte outpurstream.

Then ran protoc --include_imports --descriptor_set_out=message2.desc --
java_out=message2.proto

On the other machine:

Got the .desc file here, create a inputstream from it.

DescriptorProtos.FileDescriptorSet fileDescriptorSet =
DescriptorProtos.FileDescriptorSet.parseFrom(inputStream);

Created a descripter map and created the dynamic message:
DynamicMessage dynamicMessage = DynamicMessage.parseFrom(descriptor,
message);

when trying to print

System.out.println(" allFields.size =
"+dynamicMessage.getAllFields().size()); System.out.println("
unknownFields.tostring = "+dynamicMessage.getUnknownFields());

The fields in the Message2 are coming as the unknown fields, tried to
provide the extension registry, but not sure how to do that.

How can we dynamically parse the names and values of the fields in the
extended message2, with the .desc file. or is there other way to do
this?

Benjamin Wright

unread,
Oct 11, 2011, 2:14:14 PM10/11/11
to Protocol Buffers
It sounds like you're working in Java.

In Java you need to create, setup, and then provide an
ExtensionRegistry

1) The way you do this is you create a new one...

final ExtensionRegistry extensionRegistry =
ExtensionRegistry.newInstance();

2) then register any extensions you have by calling the appropriate
compiled proto FILE classes'
"registerAllExtensions(ExtensionRegistry)" method as such...

MyProtoFile.registerAllExtensions(extensionRegistry);

Tip: the name of this class is determined by the name of your proto
file or the "java_outer_classname" file option.

3) Then you can parse messages with this ExtensionRegistry via...

MyMessage.parse(bytes, extensionRegistry);

You probably want to create the extension registry once and then save
it for use during all parsing. You will likely need to register
multiple files worth of extension (depends on how you use extensions)
in which case simply repeat step 2 for each file using the same
ExtensionRegistry instance - all the extensions must be registered to
the same registry.

prem

unread,
Oct 11, 2011, 6:33:09 PM10/11/11
to Protocol Buffers
Thanks for the info. this solution requires the Class MyMesage be
present on the receiving side.
I was trying to just use the description-out file created by the
protoc tool.

Benjamin Wright

unread,
Oct 11, 2011, 6:59:29 PM10/11/11
to Protocol Buffers
In that case I recommend you browse through my recent discussion with
Dale Durham as I walked him through the process of using the
FileDescriptorSet's that are created when you compile using the --
descriptor_set_out flag to protoc or can obtain from another runtime
that has access to compiled protocol buffer files.

http://groups.google.com/group/protobuf/t/611309b44201a747

The only difference is that you need to register each extension
manually with the ExtensionRegistry as no "registerAllExtensions"
method is available.

You can register extensions manually from a FileDescriptor as such:

FileDescriptor fileDescriptor;
ExtensionRegistry extensionRegistry =
ExtensionRegistry.newInstance();

for(FieldDescriptor extension : fileDescriptor.getExtensions()) {
extensionRegistry.add(extension);

prem

unread,
Oct 11, 2011, 8:39:31 PM10/11/11
to Protocol Buffers
Thanks, that was the missing part, got it working with the desc file,
without the need for the MyMessage class.

on the way with the above code I got error:
java.lang.IllegalArgumentException: ExtensionRegistry.add() must be
provided a default instance when adding an embedded message extension.
at com.google.protobuf.ExtensionRegistry.add(ExtensionRegistry.java:
175)

Looked in the ExtensionRegistery.java and saw the if check for the
JavaType.MESSAGE.
so for now just added the if check like below:


for(FieldDescriptor extension : fileDescriptor.getExtensions()) {
if(extension.getJavaType() != FieldDescriptor.JavaType.MESSAGE)
{
extensionRegistry.add(extension);
}
}

What is the correct way to do it?
Reply all
Reply to author
Forward
0 new messages