Nested protobuf file folders and cmake flattening the structure.

[Alex Shaver]

When I have several folders 

top
  \ - A
  \ - B
    \ - B1
    \ - B2
...

.With various .proto files throughout, and I run CMake (3.5.2) to generate the protobuf files, it flattens the whole directory structure to just one common folder. There's a few problems with this approach. 

1. If multiple proto files have the same name (say project.foo and project.bar both have baz.proto), then the compiler chokes on the generated .pb.{h,cpp} files having the same name. 
2. Without moving the .pb.h files back into the same file structure (or a replicated structure elsewhere) as they were initially found in (and having to write a macro to do that), I can't use include statements like #include <project/foo/baz.h>
3. I've tried making each subfolder a library and then linking them together into one 'meta'-library, which fixes the first two problems, but then within protobuf the import "project/foo/baz.proto" fails because the file is in another directory. 
1. I think I could use PROTOBUF_IMPORT_DIRS to expand the search, but I may also have to mess with the include directories, specify dependencies, and so on. And even then, that may have trouble if there are circular dependencies or something.

I can't put the proto files with the code where they're used, because the protobufs are common to both C++ and Java projects. So we have to build them as an independent library that we can link against with a known header structure.

Any thoughts or improvements on this?

[Alex Shaver]

Issue with approach 3, that I don't understand: Generated header files create internal implementation functions like : 

void protobuf_InitDefaults_Baz_2eproto();

but source files using these functions are looking for:

protobuf_InitDefaults_project_2ffoo_2fBaz_2eproto();

When I compare the results against a build that used a defined makefile (not cmake), the generated header implementation matches the source call ( both use ...project_2ffoo_2fBaz_2eproto() ).

[Feng Xiao]

How did you run the protoc command? The generated code corresponds the import path of the .proto files. For example, if you have a directory like this:

root

  + project

    + foo

      + bar.proto

Invoking protoc in the root directory with:

protoc --cpp_out=. project/foo/bar.proto

will generate protobuf_InitDefaults_project_2ffoo_2fBar_2eproto.

If you invoke protoc in the project/foo directory with:
protoc --cpp_out=. bar.proto

it will generate protobuf_InitDefaults_Bar_2eproto.

The simplest way to use protoc is to always run it in the same root directory for compiling all .proto files. If you run protoc in different directories, you will need to make sure the import path is set correctly and that's complicated...

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Alex Shaver]

I'm just using the default FindProtobuf.cmake commands which have the protoc step built in to them internally. I tried mailing the cmake list for support on this, but I didn't receive a response (I'm not sure who maintains this file). That file may not have the appropriate logic to handle this situation correctly.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.

[Alex Shaver]

Solution: This doesn't seem to be either documented or intuitive, but this will solve the problem:

From the top level directory, specify your proto files relative to that directory (either listing explicitly, or file(GLOB_RECURSE proto_files RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} *.proto) if you prefer). Then set(PROTOBUF_GENERATE_CPP_APPEND_PATH OFF), which is the really non intuitive part. This flag is defaulted to on, and I would think that 'append_path' implies the kind of nested file structure I'm describing. Run everything else like normal

find_package(Protobuf REQUIRED)

file(GLOB_RECURSE proto_files RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}*.proto)

set(PROTOBUF_GENERATE_CPP_APPEND_PATH OFF)
include_directories(${PROTOBUF_INCLUDE_DIRS} ${CMAKE_CURRENT_BINARY_DIR})
protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS ${proto_files})
add_library(${PROJECT_NAME} ${PROTO_SRCS} ${PROTO_HDRS})
target_link_libraries(${PROJECT_NAME} PUBLIC ${PROTOBUF_LIBRARIES}

[Alex Shaver]

Also, this only seems to be a functionality provided in CMake 3.7.0, actually. I neglected to mention that I was experimenting with that branch when I was studying this problem.

On Monday, November 28, 2016 at 9:47:21 AM UTC-5, Alex Shaver wrote: