Trouble initializing protobufs for Android.

[Wink Saville]

I'm porting protobufs 2.0.3 to work on Android and get the following as the inital set
of errors (Note the line numbers are different because I've added some debug):


libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2017] Invalid proto descriptor for file "google/protobuf/descriptor.proto":
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf: "google.protobuf" is not a valid identifier.
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf.FileDescriptorSet.file: ".google.protobuf.FileDescriptorProto" is not defined.
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf.FileDescriptorProto.message_type: ".google.protobuf.DescriptorProto" is not defined.
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf.FileDescriptorProto.enum_type: ".google.protobuf.EnumDescriptorProto" is not defined.
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf.FileDescriptorProto.service: ".google.protobuf.ServiceDescriptorProto" is not defined.
...
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf.EnumValueOptions.uninterpreted_option: ".google.protobuf.UninterpretedOption" is not defined.
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf.ServiceOptions.uninterpreted_option: ".google.protobuf.UninterpretedOption" is not defined.
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf.MethodOptions.uninterpreted_option: ".google.protobuf.UninterpretedOption" is not defined.
libprotobuf ERROR frameworks/base/protobuf/src/google/protobuf/descriptor.cc:2020]   google.protobuf.UninterpretedOption.name: ".google.protobuf.UninterpretedOption.NamePart" is not defined.
libprotobuf FATAL frameworks/base/protobuf/src/google/protobuf/descriptor.cc:1989] CHECK failed: result != NULL:


This is my test.proto file:

syntax = "proto2";
//option cc_api_version = 1;
//option java_api_version = 1;
//option py_api_version = 1;

package frameworks_base_protobuf_android_test;

message test_1 {
  required int32 v = 1;
}

The test program is:

#include <stdlib.h>
#include "test.pb.h"

int main(int argc, char *argv[])
{
      frameworks_base_protobuf_android_test::test_1 t;

      printf("t.v()=%d\n", t.v());
      t.set_v(2);
      printf("t.v()=%d\n", t.v());
      return 0;
}

We never get to main because the failure happens during initialization and is failing in
the routine InternalBuildGeneratedFile of descriptor.cc at:

  GOOGLE_CHECK(result != NULL);

Any ideas appreciated,

Wink

[Kenton Varda]

I think you must have introduced a bug somewhere.  It may be in descriptor.cc, but it could also be some other kind of parsing or serialization bug.  Generated .pb.cc files contain encoded descriptors which are parsed at startup, and somehow the descriptor for descriptor.proto is coming up invalid.  In any case, the first error in your log there suggests that ValidateSymbolName() is being called with the string "google.protobuf", which makes no sense as "google.protobuf" is the package name, and ValidateSymbolName() is never called on the package name.

[Wink Saville]

Not surprising I introduced a bug. I'm using ustl which is a subset
of STL also using arm.

One possibility, is that there is some initialization ordering issues.
I did find one bug related to that in protobufs. The log_silencer_count_mutex_
was not being constructed before its first use. This leads me to think
maybe something similar has happened in that something else needs
to be initialized before the InternalBuildGeneratedFile is invoked.

Is that a possibility, if so what would that be.

-- Wink

[Kenton Varda]

The whole descriptor initialization process is very complicated explicitly to deal with initialization ordering issues, though bugs are certainly still possible.  I would not know exactly where to look -- if I did I would have fixed it.  :)

But the errors you mentioned don't really look like they'd be caused by incorrect initialization ordering.

Sorry, I don't have any ideas.  If you could send me your changes as a code review on codereview.appspot.com, I could take a look and see if I spot any problems.

[Wink Saville]

On Tue, Mar 31, 2009 at 6:17 PM, Kenton Varda <ken...@google.com> wrote:

The whole descriptor initialization process is very complicated explicitly to deal with initialization ordering issues, though bugs are certainly still possible.  I would not know exactly where to look -- if I did I would have fixed it.  :)

But the errors you mentioned don't really look like they'd be caused by incorrect initialization ordering.

Bummer, I was hoping for just the opposite answer:(

Sorry, I don't have any ideas.  If you could send me your changes as a code review on codereview.appspot.com, I could take a look and see if I spot any problems.

Actually, I need to diff what I've done against the virgin 2.0.3 code maybe
I've done something really stupid. I'll probably get back to you with more
questions later.

One quick question, if I remember correctly there is a call to a parser function
early on in the code. Is there a debug function which would dump the result
of the parsing?

-- Wink

[Kenton Varda]

On Tue, Mar 31, 2009 at 8:02 PM, Wink Saville <wi...@saville.com> wrote:

One quick question, if I remember correctly there is a call to a parser function
early on in the code. Is there a debug function which would dump the result
of the parsing?

None that would work at that point, unfortunately.  Message::DebugString() requires descriptors, and the place where your code is failing is during the construction of said descriptors.