Failed to parse map

[Qian Zhang]

Hi,

I have a C++ project in which there is a .proto file, and in that file, there is protobuf message which has a map field:

"map<string, string> annotations = 5;"

And the JSON file to be parsed with that .proto file has the following content:

"annotations": {
  "com.example.key1": "value1",
  "com.example.key2": "value2"
}

The .proto file is in proto2 syntax (it has "syntax = "proto2";" at the beginning) and I am using protobuf-3.3.0, the compilation succeed, but I found the annotation map always has only one entry, and both the entry's key and value are "" which is obviously not correct.

Any suggestions?

[Adam Cozzette]

It looks to me like your JSON syntax is right. Could you post the C++ code you are using to parse the JSON file?

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Qian Zhang]

Hi Adam,

We are using picojson to parse the JSON file.

After more debugging, I think I have found the root cause: In the code here, we are trying to find a field by the key of an entry in a map, obviously it will fail since the key of any map entries is not in the protobuf message definition (the .proto file), and to avoid finding field by a map entry's key, in this method, we need to check if "field->is_map()" is true which is currently missed, if the field is a map, then we should use the reflection to construct a map message, however I do not find a method in protobuf to do that, I find a lot of "Addxxx()" methods (e.g., "AddString()", "AddInt64()", etc.), but not a method for adding a map. There is a method "InsertOrLookupMapValue()", but that is a private method which I can not call in my code.

So can you please suggest how to add a map message? Thanks!

Regards,

Qian Zhang

[Adam Cozzette]

Doing reflection on a map is a little bit tricky, but the way to do it is to treat it as a repeated field since that is how it is actually represented on the wire. A map is a stored as a repeated message field, where in each message the key is field 1 and the value is field 2.

[Qian Zhang]

Thanks Adam.

Can you please let me know which API I can use to do reflection on a map?

Regards,

Qian Zhang

[Adam Cozzette]

Here's an example of how to do it: http://google3/experimental/users/acozzette/map_reflection_test.cc?rcl=158157727 It's a bit clunky because you just have to reflect on the map field like you would a repeated field, but it works.

[Qian Zhang]

It seems the URL http://google3/experimental/users/acozzette/map_reflection_test.cc?rcl=158157727 is not working, I can not access it :-(

Regards,

Qian Zhang

[Adam Cozzette]

Hmm, that's strange. If that's not working, you can always just look at the file in a Piper client, though.

[Adam Cozzette]

Oops, sorry about that. Somehow I had gotten mixed up and thought this was our Google-internal mailing list, so that would explain why the link was not accessible.

Here is the example code I was trying to link to:

The proto file:

syntax = "proto2";

package acozzette;

message MapMessage {

  map<string, int32> name_to_number = 1;

}

The C++ code for adding a map entry via reflection:

void SetMapEntry(google::protobuf::Message* entry) {

  const google::protobuf::FieldDescriptor* key_field =

      entry->GetDescriptor()->FindFieldByNumber(1);

  const google::protobuf::FieldDescriptor* value_field =

      entry->GetDescriptor()->FindFieldByNumber(2);

  const google::protobuf::Reflection* reflection = entry->GetReflection();

  reflection->SetString(entry, key_field, "xyz");

  reflection->SetInt32(entry, value_field, 123);

}

TEST(MapReflectionTest, MapReflection) {

  MapMessage message;

  const google::protobuf::FieldDescriptor* repeated_field =

      message.GetDescriptor()->FindFieldByName("name_to_number");

  const google::protobuf::Reflection* reflection = message.GetReflection();

  google::protobuf::MutableRepeatedFieldRef<google::protobuf::Message>

      repeated_ref =

          reflection->GetMutableRepeatedFieldRef<google::protobuf::Message>(

              &message, repeated_field);

  std::unique_ptr<google::protobuf::Message> entry(repeated_ref.NewMessage());

  SetMapEntry(entry.get());

  repeated_ref.Add(*entry);

  EXPECT_EQ(1, message.name_to_number().size());

  EXPECT_EQ(123, message.name_to_number().at("xyz"));

}

[Qian Zhang]

Thanks Adam, I have tried it, it works perfectly!!!

Regards,

Qian Zhang