GSON Serialization of Nested Protobuf Messages/Classes

[markk...@gmail.com]

Hi there,

I'm hitting a wall while attempting to use GSON to serialize a nested set of Protobuf GeneratedMessage objects.

Looking at the rough example at https://code.google.com/p/google-gson/source/browse/trunk/proto/src/main/java/com/google/gson/protobuf/ProtoTypeAdapter.java?r=613 I've managed to implement a nearly complete JsonSerializer<GeneratedMessage> that supports the serialization of Protobuf primitives, enums, and repeated values.

However, I can't figure out how to tweak my JsonSerializer to support nested Protobuf messages.  For instance:

message User {
  required String userId = 1;
  required String name = 2;
}

message Session {
  required String sessionId = 1;
  required User user = 2; // "Nested".. references message User above
}


When compiled using 'protoc', these result in the following Java classes (highly distilled):

import com.google.protobuf.GeneratedMessage;

public final class User extends GeneratedMessage {
  private String userId;
  private String name;
}

public final class Session extends GeneratedMessage {
  private String sessionId;
  private User user;
}


I want to write a generic serializer for "GeneratedMessage".  Ignoring deserialization for the moment, my JsonSerializer<GeneratedMessage> looks something like this:

public final class GeneratedMessageAdapter implements JsonSerializer<GeneratedMessage> {

  @Override
  public JsonElement serialize(GeneratedMessage src, Type typeOfSrc, JsonSerializationContext context) {
    JsonObject ret = new JsonObject();
    Map<Descriptors.FieldDescriptor, Object> fields = src.getAllFields();
    for (Map.Entry<Descriptors.FieldDescriptor, Object> fieldPair : fields.entrySet()) {
      final Descriptors.FieldDescriptor desc = fieldPair.getKey();
      final Object value = fieldPair.getValue();
      if (desc.isRepeated()) {
        // Handle Protobuf "repeated" value as a JsonArray
        ret.add(desc.getName(), new JsonArray(...));
      } else if (desc.getJavaType().equals(JavaType.ENUM)) {
        // Handle Protobuf "enum" value as a JsonPrimitive
        ret.add(desc.getName(), new JsonPrimitive(...));
      } else if (desc.getJavaType().equals(JavaType.MESSAGE)) {
        // Nested Protobuf GeneratedMessage
        // ***Need to somehow re-invoke my GeneratedMessageAdapter to serialize the nested object??***
      } else {
        // Handle Protobuf primatives (everything else)
        ret.add(desc.getName(), context.serialize(value));
      }
    }
    return ret;
  }

}


And, I'm registering my adapter using the following:

GsonBuilder builder = GsonBuilder().serializeNulls();
builder.registerTypeAdapter(GeneratedMessage.class, new GeneratedMessageAdapter());


It seems my problem is in the else if (desc.getJavaType().equals(JavaType.MESSAGE)) conditional above.

Calling context.serialize(value) on this nested GeneratedMessage object seems to invoke the default object serializer, and not my GeneratedMessage registered type adapter.  As such, the serialized JSON for any nested GeneratedMessage objects contains Protobuf internal fields rendered in a way that's unusable to my application in the browser.

Further, explicitly calling context.serialize(value, GeneratedMessage.class)to force the issue results in a classic StackOverflowError (expected, according to documentation but I tried).

Lastly, I tried implementing my own new JsonSerializationContext(){...} inline and invoking .serialize on that using a new Gson instance just for this one case... thinking I was clever enough, but no dice.

In the end, pretty sure this problem isn't specific to Protobuf generated classes.  I conjecture anyone would have the same problem with these, trying to write a generic serializer for Bar:

public abstract class Bar {}

public class Foo extends Bar {}
public class Baz extends Bar {
  private Foo foo;
}


If anyone out there has advice or thoughts on how to address this, I would be forever grateful.  Thanks!

-Mark

[markk...@gmail.com]

Sigh, looks like I'm being bitten by this https://code.google.com/p/google-gson/issues/detail?id=137

[markk...@gmail.com]

After looking at this for several hours in a debugger, I realized what I was doing wrong.  Had nothing to do with GSON, but rather my own programming error.

It turns out that generated Protobuf classes, at least in Java, are self-referential.  That is, if you set a breakpoint and dig into the object structure you'll find that the Protobuf object contains fields which are pointers to metadata about the object, that then also happen to point back to the actual Protobuf object itself.

public final class FooProto extends GeneratedMessage {

  private static class FooProtoMetadata {
    private final FooProto self_; // Doh!
  }

  // ... other fields here

  public final FooProtoMetadata metadata_;

}

So in the end, my grief here was caused by attempting to recursively serialize the same Protobuf object, forever.

Classic.

-Mark

On Wednesday, August 13, 2014 8:49:37 AM UTC-7, markk...@gmail.com wrote: