Storing Proto Files in a Separate Directory

[Nicholas Bunn]

Hi everyone,

Up until now I've been structuring my project by containing a services' proto files in the same folder as the service itself, as below:

- serviceOne

    - serviceOne.py

    - protos

        - serviceOne.proto

        - serviceOne_pb2.py

        - serviceOne_pb2_grpc.py

        - serviceOne.pb.go

        - serviceOne_grpc.pb.go

This would be for a server written in Python, being called by a client written in Go. This has been working perfectly fine but I'm trying some different project structures to try and get around the client needing all of the server's generated code in order to get the proto files.

I'm trying out the following file structure as a possible alternative:

- protos

- serviceOneProto

- v1

- serviceOne.proto

- python

- serviceOne_pb2.py

        - serviceOne_pb2_grpc.py

- go

- serviceOne.pb.go

        - serviceOne_grpc.pb.go

- services

- serviceOne

- serviceOne.py

-serviceTwo

-serviceTwo.go

Here, the proto files are all contained within their own directory, with the generated proto files being stored in language-specific folders.

I just wanted to find out if there are any glaring reasons against following a project structure like this and, if not, what would be the best way to importing the proto files (should one create a module for each service's proto files)? I've been fighting around with some ModuleNotFound errors trying to get the Python service to run with this structure - so any assistance would be greatly appreciated!

[Nicholas Bunn]

I suppose another method to separating the generated proto files from the server would be to generate the proto files directly into the relevant service directory. So serviceOne.proto would still live in the serviceOne directory, along with the generated Python files as these will be used by the Python server; but the .pb.go files would be generated into the Go service directory as they will be used by the Go client. This keeps the proto definitions close to the related service but gets me around my original issue.

[Amit Saha]

Your idea of using a separate Python package and a go module for the
generated code is likely the best approach here. That is, the
following directory structure:

- service-defs - here you have the protobufs. The generated code will
live in their own sub-directories and will be version controlled
(perhaps in a separate repo). However, you should publish the Python
package for the Python code and a go module for the go code from here.
For local development, you of course don't need to publish your
packages. The key is that the generated code should be accessed as
packages and the generated code should live in a single place.

- server - Your Python server code which imports the package
containing the generated code, version controlled and perhaps a
separate repo

- client - Your Go client which imports the Go module containing the
generated code, version controlled and perhaps a separate repo

I mention the word "perhaps" with a separate repo since, if you are
just starting out and want to keep things simple, you can have all of
them in a single repo, but the accessing mechanisms should be via
Python package and go modules. That way, if you have to move them to
separate repos, your code doesn't need changing. Your local
development workflow may need changing, but otherwise you will be
good.



>
> --
> You received this message because you are subscribed to the Google Groups "grpc.io" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/1d6ecb8e-1c27-43ef-8491-fd65de65d111n%40googlegroups.com.