grpc Equivalent of Map<String, Object>

[qureshi....@gmail.com]

Hello Experts,

I was curious to find out if there is any protobuf which can be used equivalent of Java Map<String, Object>. Some protobuf contract which can be used as

map<string, object> attributes = 1;

Any help would be greatly appreciated.

Thanks,

-Imtiyaz 

[Andreas Pillath]

Maybe you could try:

import "google/protobuf/any.proto";

message AnyMessage {

  map<string,google.protobuf.Any> attributes = 1;  

}

https://developers.google.com/protocol-buffers/docs/proto3#any

[qureshi....@gmail.com]

Thanks Andreas for you response. Is there any reference in Java which I can refer to implement this? Appreciate your help

[zun...@humu.com]

A few alternatives, depending on your needs: google.protobuf.Any is a bit tricky to use, and is fairly expensive on the wire since it serializes the name of the protobuf type as well as its contents:

* If you already know the type of the objects at the receiver side, you can use

map<string, bytes> attributes = 1;

and just store the byte-serialized proto value in the map. This is the "old-fashioned" way to do it, and is still the most wire-efficient and code-simple solution if you know the type ahead of time.

* If you don't, a good alternative to using Any is to roll your own version of it:

message Object {

  string protoType = 1;

  bytes value = 2;

}

map<string, Object> attributes = 1;

Then on the client side, get the class object from protoType using the PB symbol database (syntax varies by language), and decode value using that. 

This may be better than Any, because Any is not just a somewhat recent addition to proto, it has some rather surprising behavior: the proto type you pass to it needs to be the URL where a schema descriptor is being served, and sometimes if you try to pass an Any through various services (like Stackdriver logging, as I found to my dismay) it will not only try to look up that descriptor, but fail the entire RPC if that doesn't work. This is behavior which almost nobody actually needs, and requires you to do a lot of work (exposing an API publicly) to support, so I frankly recommend skipping it unless you have a good reason.

* One thing I don't recommend is something like

message Object { }

map<string, Object> attributes = 1;

and then using proto extensions to extend Object. This was the old (proto2) way to do this, but it got taken out of proto3 because it's a weird maintenance nightmare, the wire formats made the proto code way more complex, and it's prone to Interesting Surprises in several languages, especially Java. Proto extensions aren't quite the work of the Devil, but they're not too far off.

On Thursday, June 14, 2018 at 6:31:46 AM UTC-7, Andreas Pillath wrote:

[Carl Mastrangelo]

We don't use maps frequently in protobuf, but we do use Any.  You can see some example Java code on how to do it in the Channelz service implementation:

https://github.com/grpc/grpc-java/blob/f5f560a/services/src/main/java/io/grpc/services/ChannelzProtoUtil.java#L271

[Benjamin Krämer]

What I use for this purpose is a own Value type inspired by Value from struct.proto (https://github.com/google/protobuf/blob/master/src/google/protobuf/struct.proto#L63). In addition, I have some C# extension methods for converting Value to Object for easier usage. This is a lot cheaper on the wire than using Any and most of the time I only have native values. And for non-natives I can choose to use bytes if the type is known or Any if it's unknown.

message Struct {
  // Unordered map of dynamically typed values.
  map<string, Value> fields = 1;
}

message Value {
  // The kind of value.
  oneof kind {
    // Represents a null value.
    NullValue null_value = 1;
    // Represents a double value.
    double double_value = 2;
    // Represents a float value.
    float float_value = 3;
    // Represents an int32 value.
    int32 int32_value = 4;
    // Represents an int64 value.
    int64 int64_value = 5;
    // Represents an uint32 value.
    uint32 uint32_value = 6;
    // Represents an uint64 value.
    uint64 uint64_value = 7;
    // Represents a sint32 value.
    sint32 sint32_value = 8;
    // Represents a sint64 value.
    sint64 sint64_value = 9;
    // Represents a fixed32 value.
    fixed32 fixed32_value = 10;
    // Represents a fixed64 value.
    fixed64 fixed64_value = 11;
    // Represents a sfixed32 value.
    sfixed32 sfixed32_value = 12;
    // Represents a sfixed64 value.
    sfixed64 sfixed64_value = 13;
    // Represents a boolean value.
    bool bool_value = 14;
    // Represents a string value.
    string string_value = 15;
    // Represents a bytes value.
    bytes bytes_value = 16;
    // Represents an Any value.
    google.protobuf.Any any_value = 17;
    // Represents a structured value.
    Struct struct_value = 18;
    // Represents a repeated `Value`.
    ListValue list_value = 19;
  }
}

enum NullValue {
  // Null value.
  NULL_VALUE = 0;
}

message ListValue {
  // Repeated field of dynamically typed values.
  repeated Value values = 1;
}