how best to do a cross-platform .proto file

[j...@joeshaw.org]

Hi,

I have a .proto file that has a fair number of gogoprotobuf-specific options inside it. The top of the file looks like this:

// Go-specific options
option go_package = "protobuf";
import "code.google.com/p/gogoprotobuf/gogoproto/gogo.proto";
option (gogoproto.sizer_all) = true;
option (gogoproto.marshaler_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.populate_all) = true; // needed by testgen
option (gogoproto.equal_all) = true; // needed by testgen
option (gogoproto.testgen_all) = true;

I'd like to share this .proto file across a few different projects in different languages including Python, Java, C#, and Obj-C. But obviously I'd like for people to not need to import gogo.proto if they're not using Go.

My first thought was to move the Go-specific stuff into a separate foo-go.proto file that imports foo.proto. If the import is not marked "public" then foo-go.pb.go is mostly empty. If I do mark it "public" then foo-go.pb.go has the definitions, but it tries to do 'import foo "foo.pb"' which is also not what I want.

Ideally I'd like for the foo.proto file to be treated as thought it were embedded inside the foo-go.proto file when the code is ultimately generated. Is there a better way to do that?

Thanks,
Joe

[Walter Schulze]

On 7 August 2014 03:05, <j...@joeshaw.org> wrote:

Hi,

I have a .proto file that has a fair number of gogoprotobuf-specific options inside it.  The top of the file looks like this:

// Go-specific options
option go_package = "protobuf";
import "code.google.com/p/gogoprotobuf/gogoproto/gogo.proto";
option (gogoproto.sizer_all) = true;
option (gogoproto.marshaler_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.populate_all) = true; // needed by testgen
option (gogoproto.equal_all) = true;  // needed by testgen
option (gogoproto.testgen_all) = true;

I'd like to share this .proto file across a few different projects in different languages including Python, Java, C#, and Obj-C.  But obviously I'd like for people to not need to import gogo.proto if they're not using Go.

Protobufs are cross language compilable.
This is why I chose to extend the descriptor.
They are not importing Go they are importing protobuf extensions for the descriptor.

There are other extensions for fields and messages that are not so easily abstracted away.
Making the protobufs with gogoproto extensions usable without importing gogo.proto would require you to remove all mentions of gogoproto from the proto file.
I would do this by parsing the proto file into a descriptor and then writing code to remove all extensions referring to gogoproto.
But I wouldn't do this.
This seems like quite a lot of work for something that is not a problem.

We use it a lot with java, without any problems.

Here is a link to a Dockerfile which shows how to compile C++ in two ways
https://code.google.com/p/gogoprotobuf/source/browse/dockertest/cpp/Dockerfile
The same person who got me to fix the C++ compatibility is also using it with Python.
Here is a liink to why I had to make some breaking changes to support C++
https://groups.google.com/forum/#!topic/gogoprotobuf/1E4BIQF4OYM
 

My first thought was to move the Go-specific stuff into a separate foo-go.proto file that imports foo.proto.  If the import is not marked "public" then foo-go.pb.go is mostly empty.  If I do mark it "public" then foo-go.pb.go has the definitions, but it tries to do 'import foo "foo.pb"' which is also not what I want.

I have actually not played around with import public before.

It seems gogoprotobuf does not actually support it for its extra code generation.
I have added an issue
https://code.google.com/p/gogoprotobuf/issues/detail?id=22

Unfortunately I think your idea does not use public for its intended purpose.

The idea is that you should not break any code if you move your proto file and create a new file at the old location and public import the moved one.

This would mean that the generated code should be usable from the old location.

So the code generation flags defined in the original proto at the new location should hold in the new proto at the old location.

I think code generation flags defined in the new proto at the old location should only hold for new messages defined in that file.

Btw I think this is public import feature is a hack.
I mean oldlocation.MyMessage is a different type to newlocation.Message eventually these types are going to interact with each other in some code that expects that type to be passed to a function and you are going to run into compilation errors in some programming languages.

But that is my naive opinion.

 

Ideally I'd like for the foo.proto file to be treated as thought it were embedded inside the foo-go.proto file when the code is ultimately generated.  Is there a better way to do that?

I think you should just import the protos correctly.

But I don't know what you office politics are like, so maybe maybe maybe you should consider using sed just before your protoc step.

This will only work if you are only using file extensions.

I hope this helps.

 

Thanks,
Joe

--
You received this message because you are subscribed to the Google Groups "gogoprotobuf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gogoprotobuf...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.