Exporting .proto vs generated .pb files

[prot...@gmail.com]

What is everyone doing in terms of making services available to other applications/libraries? We're debating between either:

1. have the service generate only the .pb for the language it needs (so it can implement the various endpoints), and export only the .proto file, thereby leaving it up to the various clients to generate the .pb files for their respective languages from exported .proto file.

2. have the service also generate .pb files for other languages, and export the generated files directly instead of the .proto file.

With #1,  the obvious downside though is that the clients now all need to have protoc installed (though often times we need to regenerate them anyways when we upgrade grpc-go or some other language bindings). It's not the easiest dependency to manage.

With #2, while convenient at first glance, has issues in that it's possible the service-generated .pb file is incompatible with the expectations of the client's installed grpc library. In fact we've seen this with grpc-go. It also seems cleaner to have each client/server only generate the bindings it needs, as opposed to leaving it up to the service to generate the bindings for all X possible future languages. In fact we'd need to do one of these when incompatibilities arise:

a) force the server to upgrade to a new version of grpc just to generate a new .pb file that can satisfy a client with a newer version of grpc

b) force the client to vendor an older version of grpc.

What's the recommended best practice on this?

[Nicolas Noble]

My personal recommendation would be to enforce generation of source code as much as possible, while retaining a decent amount of sanity. Shipping generated code can very quickly become a synchronization problem, but right now, the generation tools are too difficult to acquire. We are working on making this better, but we are not there yet.

So, basically, until there is a better and easier story to have the protoc and plug-ins distributed, I would push both files for now, so that people can regenerate if they want to, but only if they want to. Once protoc and the plug-ins have more availability so that it's not a burden for the users to acquire them, yank the generated files out.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d66b5fe5-a2de-4b51-b784-6a2d4c4b6039%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[prot...@gmail.com]

Thanks. We have complete control over our build system for now, so we'll leave it up to the clients to generate the necessary definitions. Ideally each language can generate the necessary pb files entirely independent of the C++ code, but that's a long ways coming.

[Eric Anderson]

For Google-provided services, in general, we're planning on shipping generated code while also having the protos available. The generated code would be what is used most and would have language-specific packaging available to make the experience easy. The protos would still be available for general documentation but you could also end up making your own protos that depend on Google-service protos.

There can be exceptions, like C++ for instance. Also, since we haven't reached 1.0, there could be some possibilities for breakage. We're actually committed to preventing breakages, but if something really important came up it's conceivable we would break the API.

Java is another special case since codegen for protobuf has never been stable. That should be fixed once protobuf 3.0 is released (not a beta).

After 1.0, the general expectation is that small projects will just provide protos, since it can be a pain to generate code for all languages and make packages available. Large companies, where it makes sense to overcome the overhead, will publish the code-generated code as well.

The tooling is still in its infancy, but https://github.com/grpc/grpc-tools provides scripts to generate packages for various languages. We should see the tooling mature with time.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/fc8a55bc-363f-47a8-b466-c49bd74fc8b8%40googlegroups.com.