Is there a reliable method to dynamically identify and lookup a message from both the Java and Java script API?

[Keith Woods]

I'm trying to create a wrapper DTO whereby the inner payload could be anything. The main consumer will be a router that has to handle the deserialization and fan the message out to receiving services. To do this I need a key to effectively wrap and unwrap proto objects in a layer that knows nothing of the inner payload type. I did look at using Any to get this to work, however that requires you have a handle to the class of the message you need to unpack. I worked around this creating a similar construct of my own, AnyDto.

I created these protos to do the job:

syntax = "proto3";

option java_multiple_files = true;

package proto3spike.dtos;

message EnvelopeDto {

  string operationName = 1;

      AnyDto payload = 2;

}

message AnyDto {

   string canonicalName = 1;

   bytes value = 2;

}

The following utility to wrap and unwrap the AnyDto into a Message instance

package proto3spike;

import com.google.protobuf.InvalidProtocolBufferException;

import com.google.protobuf.Message;

import com.google.protobuf.MessageLite;

import proto3spike.dtos.AnyDto;

public class AnyDtoMapper {

    public static AnyDto mapToAnyDto(Message payload) throws InvalidProtocolBufferException {

       String canonicalName = payload.getClass().getCanonicalName();

       return AnyDto.newBuilder()

               .setCanonicalName(canonicalName)

               .setValue(payload.toByteString())

               .build();

   }

    public static Message mapFromAnyDto(AnyDto dto) throws Exception {

       Class c = Class.forName(dto.getCanonicalName());

       MessageLite defaultInstance = com.google.protobuf.Internal.getDefaultInstance(c);

       return (Message)defaultInstance

               .getParserForType()

               .parseFrom(dto.getValue());

   }

}

When using mapFromAnyDto() higher order code needs to cast to a specific Message instance. For now I'm happy to deal with that, effectively some routing on the operationName field will get the message to the right place where the case can be performed in a realisable way. 

This works nicely with java to java communication. 

However in Javascript, when I come to wrap and unwrap the message I'm left with no reliable means to map AnyDto.canonicalName to the generated proto objects. 

It does look like the generated JS protos contain similar namespacing to the corresponding class (and thus canonical name) in the java space. For example, in the generated JS PB file:

/**

* @fileoverview

* @enhanceable

* @public

*/

// GENERATED CODE -- DO NOT EDIT!

var jspb = require('google-protobuf');

var goog = jspb;

var global = Function('return this')();

goog.exportSymbol('proto.proto3spike.dtos.AnyDto', null, global);

goog.exportSymbol('proto.proto3spike.dtos.EnvelopeDto', null, global);

/**

* Generated by JsPbCodeGenerator.

* @param {Array=} opt_data Optional initial data array, typically from a

* server response, or constructed directly in Javascript. The array is used

* in place and becomes part of the constructed object. It is not cloned.

* If no data is provided, the constructed object will be empty, but still

* valid.

* @extends {jspb.Message}

* @constructor

*/

proto.proto3spike.dtos.EnvelopeDto = function(opt_data) {

 jspb.Message.initialize(this, opt_data, 0, -1, null, null);

};

goog.inherits(proto.proto3spike.dtos.EnvelopeDto, jspb.Message);

if (goog.DEBUG && !COMPILED) {

 proto.proto3spike.dtos.EnvelopeDto.displayName = 'proto.proto3spike.dtos.EnvelopeDto';

}

//  ... rest of the file ...

As you can see in debug mode there is a displayName on the message objects, additionally the messages get exported to a global namespace that somewhat matches the java package (and thus canonical name). 

Using the canonical name of the java Message.class now seems like a hack. 

Is there a reliable method to identify and lookup a message from both the Java and Java script API so I can dynamically serialise/deserialise it? 

If not, can I somehow extend the generated JS to add a custom key (i.e. the canonical name) to the generated JavaScript code?

Thanks for any help

Keith

[Feng Xiao]

Mapping from generated class name to a protobuf message type is generally not a good idea. It can be easily broken without notice when you update your proto file (e.g., move a message from one .proto file to another, or just set a java_multiple_files option). Also languages that don't have reflection support (like C++) won't be able to parse your data created this way. I would suggest use a more language-natural canonical name and build a custom mapping yourself (e.g., a Map<String, Message> in Java).

 

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Keith Woods]

Thanks for the reply. I agree, mapping from a generated class name isn't a good idea. However just re-reading my initial post, while I mentioned my use of the canonical name of the java Message, in fact this is really just the proto package name. It would be nice if metadata for the package name (as defined by the 'package' keyword) was available on both the Java side and the generated JS proto messages to support more dynamic use cases. To some degree it is available as the package name corresponds to the java class's canonical name, and on the JS side it corresponds to the namespace the messages exists on the global proto object. It would be nice if this was more formal (for example each message having typing information/metadata). 

[Feng Xiao]

On Mon, Jun 6, 2016 at 2:07 AM, Keith Woods <devs...@gmail.com> wrote:

Thanks for the reply. I agree, mapping from a generated class name isn't a good idea. However just re-reading my initial post, while I mentioned my use of the canonical name of the java Message, in fact this is really just the proto package name. It would be nice if metadata for the package name (as defined by the 'package' keyword) was available on both the Java side and the generated JS proto messages to support more dynamic use cases. To some degree it is available as the package name corresponds to the java class's canonical name, and on the JS side it corresponds to the namespace the messages exists on the global proto object. It would be nice if this was more formal (for example each message having typing information/metadata). 

In C++ we have a global generated_pool()/generated_factory() which can map a canonical proto type name to a proto instance. It's not implemented in Java though. I'm not familiar with JS but it's likely not available there as well. +haberman

[Josh Haberman]

HI Keith,

Can you post more about what your router needs to do with the wrapped message? What prevents you from just tunneling the serialized message as bytes? What do you gain by knowing its type more specifically and unpacking it?