Managing Shared & Sharing proto definitions & compile packages (go)

[Brett Mostert]

Hi,

I am busy with a side project to learn more about the GRPC ecosystem. I want to check if I am headed down the right path when it comes to vendor proto/gen-proto.

Current Project Setup.

• Mono Repo for proto definitions
  • The build automatically builds proto definitions and push them to a remote repo. Example - https://github.com/brettmostert/trple-proto-product-go
• Implementation Project (server and client)
  • Uses the lib/module from https://github.com/brettmostert/trple-proto-product-go
  • Repo of Implementation https://github.com/brettmostert/trple

So from what I can tell the above is a common setup for sharing "internal" proto definitions and compiled/generated proto libs.

The real question is when using a "vendor"'s proto definitions Example https://github.com/googleapis/go-genproto/tree/main/googleapis/type

I believe the setup would be to

1. Clone the proto definition repo and import the proto files I need in my proto project i.e. money.proto. So that my proto's can reference the vendor ones.
2. Clone the proto generated repo and replace the vendors module URL with my forked version using `go mod edit -replace="google.golang.org/genproto=github.com/brettmostert/go-genproto@3a66f56" `

So in summary...

• Create your OWN proto repo
• FORK any proto repo you want to use
• Generate the code for the FORKED Repo
• Generate the code for the your OWN
  • which is dependant on the generated code from the FORKED REPO
  • so replace the original module url with your forked version (go)
• Import your OWN generated module in the implementation project

Thanks in advance... The repo's linked for triple are work in progress until I get all the scaffolding done. I do know about buf.build, its really nice, but I want to get it working without the buf.build registry and once I understand it, I may move onto some nice tooling like their registry.

Thanks,

Brett

[dfa...@google.com]

This is probably a better question for the protobuf team than the gRPC team.

What you described seems like it would work, if my understanding of go.mod is correct.  The concern with using generated (pb.go) files from anywhere but a common source location is that the proto library will panic if multiple definitions of the same message are found.  However, I believe using the replace directive will affect all libraries you are depending upon to reference your vendored copy, and it should be okay.  The stubs generated for gRPC do not have this potential problem, as there is no global registry of service definitions.

Let us know if you run into any problems with your approach.

Thanks,

Doug