Google.Protobuf.InvalidProtocolBufferException: Unknown field: globalIndex

[Ryan O'Shea]

Hi all,

Starting this morning EDT, we began receiving intermittent errors when attempting to parse the result JSON of an AsyncBatchAnnotateFiles request using JsonParser from Google.ProtoBuf. The JSON returned from AsyncBatchAnnotateFiles varies from request to request, namely in that occasionally the response contains a "globalIndex" property belonging to the Words objects. When globalIndex is included in the result JSON, the parsing of the result JSON as an AnnotateFileResponse fails; otherwise, it succeeds. There doesn't seem to be any pattern to when the service returns the error-producing JSON vs. the JSON we've been used to seeing (without globalIndex), though both are valid JSON. The same document can produce both working and non-working JSON results using the same API calls.

The line in our C# project producing the exception:

JsonParser.Default.Parse<AnnotateFileResponse>(json)

And the exception itself produced when the JSON contains the globalIndex fields:

Google.Protobuf.InvalidProtocolBufferException: Unknown field: globalIndex

   at Google.Protobuf.JsonParser.Merge(IMessage message, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.ParseSingleValue(FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.MergeRepeatedField(IMessage message, FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.Merge(IMessage message, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.ParseSingleValue(FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.MergeRepeatedField(IMessage message, FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.Merge(IMessage message, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.ParseSingleValue(FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.MergeRepeatedField(IMessage message, FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.Merge(IMessage message, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.ParseSingleValue(FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.MergeRepeatedField(IMessage message, FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.Merge(IMessage message, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.ParseSingleValue(FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.MergeField(IMessage message, FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.Merge(IMessage message, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.ParseSingleValue(FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.MergeRepeatedField(IMessage message, FieldDescriptor field, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.Merge(IMessage message, JsonTokenizer tokenizer)

   at Google.Protobuf.JsonParser.Merge(IMessage message, TextReader jsonReader)

   at Google.Protobuf.JsonParser.Parse[T](TextReader jsonReader)

   [<our code>]

A portion of the offending JSON, with "globalIndex" property highlighted:

I'm posting here (rather than StackOverflow) because this seems to be an issue with the AsyncBatchAnnotateFiles producing different results on repeated calls to annotate the same single file. 

For the Google.Cloud.Vision.V1 Nuget package, we originally saw this issue this morning on v1.3.0, so we upgraded to latest v1.6.0 under the assumption that a breaking change had been made to the AnnotateFileResponse schema, but we are still seeing the issue using the latest v1.6.0. The version of Google.Protobuf is 3.8.0, which is not the latest (3.9.1), but it was the version we were automatically upgraded to when upgrading Google.Cloud.Vision.V1 to 1.6.0.

We would appreciate any insight into this, especially if we're doing something wrong here. Is this behavior expected?

Thanks,

Ryan

[Duane Chen]

Hi Ryan,

There was a problem during our new release, which is promptly rolled back. The issue should be gone now. Thank you and apologies for the inconvenience.

Duane

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Cloud Vision Discussion Google Group (cloud-visi...@googlegroups.com) to participate in discussions with other members of the Google Cloud Vision community and the Google Cloud Vision Team.
---
You received this message because you are subscribed to the Google Groups "cloud-vision-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-vision-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-vision-discuss/a61947d7-fe81-4cf9-8f42-a06d6ab78044%40googlegroups.com.

[Ryan O'Shea]

Hi Duane,

Thanks for the prompt reply. We're happy to hear it was a temporary issue. We have two follow-up items:

1. Does Cloud Vision have a status page we can use to check in on issues like these without posting here? I don't see it listed on the GCP status page.
2. Are there any release notes or support tables published for the client libraries (in our case, for .NET)? We tried upgrading the client libraries from 1.3.0 to 1.6.0 in the hopes that it'd solve this issue while it was happening, but we did so blindly as we couldn't find any notes on what changed between versions or whether 1.3.0 was still supported.

Thanks again,

Ryan

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-vision-discuss+unsub...@googlegroups.com.

[Duane Chen]

Hi Ryan,

We don't have a public facing status dashboard today but will definitely look into adding one for the future.

For the release notes on your C# library, I found the relevant Github commit https://github.com/googleapis/google-cloud-dotnet/commit/ec4e5c2e480bf5b11f1899e888726ee645a278cf which has the changelog.

Thank you!

Duane

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-vision-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-vision-discuss/a61947d7-fe81-4cf9-8f42-a06d6ab78044%40googlegroups.com.

--

© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Cloud Vision Discussion Google Group (cloud-visi...@googlegroups.com) to participate in discussions with other members of the Google Cloud Vision community and the Google Cloud Vision Team.
---
You received this message because you are subscribed to the Google Groups "cloud-vision-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-vision-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-vision-discuss/fd9f5f7c-6ae7-4458-a596-8185418c3241%40googlegroups.com.

[Ryan O'Shea]

We've begun seeing a similar issue in the last 48 hours. This time, the unknown field is isEmbeddedText. We haven't changed any client library versions since the original message in this thread from Sep 3.

Google.Protobuf.InvalidProtocolBufferException: Unknown field: isEmbeddedText

Any advice you can offer here?

Thanks,

Ryan

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-vision-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-vision-discuss/a61947d7-fe81-4cf9-8f42-a06d6ab78044%40googlegroups.com.

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Cloud Vision Discussion Google Group (cloud-visi...@googlegroups.com) to participate in discussions with other members of the Google Cloud Vision community and the Google Cloud Vision Team.
---
You received this message because you are subscribed to the Google Groups "cloud-vision-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-vision-discuss+unsub...@googlegroups.com.

[cloud-vision-discuss]

Hi Ryan,

Apology for breaking your pipeline - that was an experimental feature released prematurely into production.

However if you're using C# we recommend you ignore unknown field (https://github.com/protocolbuffers/protobuf/blob/master/csharp/src/Google.Protobuf/JsonParser.cs#L1008) as otherwise the code won't be future compatible for any new features we release. Could you please give it a try?

Thanks.

[Ryan O'Shea]

Thanks for making us aware of that option in the JsonParser. 

Due to the delay in receiving a response, we updated our pipeline to parse the JSON itself rather than relying on the AnnotateFileResponse class provided by the client libraries. Most of our logic was already working with the JSON manually as we'd began our project using the REST API rather than the client library, so this wasn't too difficult a change to make.