Using cmake with protobuf files in subdirectory and using cmake build directory specifies invalid output directory

[cru...@gmail.com]

When using cmake to use protobufs, there's no way to have protoc create the output files in the cmake build directory unless the build directory is a direct subdirectory from the source directory.

For example, I have source code that resides in directory ${SRCDIR} with the protobuf files in ${SRCDIR}/protobuf. The cmake file (for a variety of reasons) is not located in ${SRCDIR} but in ${SRCDIR}/cmake_server/DMSO. I'm building in ${SRCDIR}/cmake_server/DMSO/cmake_build_vs2017 using Visual Studio 2017.

${SRC_DIR}/

  - protobuf/

    pb_hla_utilities.proto

    pb_messages.proto

  - cmake_server/

    - DMSO/

      CMakeLists.txt

      - cmake_build_vs2017/

        - protobuf_build/       <== What I want

Here's the relevent code in my CMakeLists.txt file:

set(PROTBUF_FILES
    ${SRCDIR}/protobuf/pb_messages.proto
    ${SRCDIR}/protobuf/pb_hla_utilities.proto
  )

set(protobuf_DIR <path_to_protobuf_install>/cmake)
find_package(protobuf REQUIRED CONFIG)

protobuf_generate(
  APPEND_PATH
  TARGET ${LOCAL_EXE_NAME}
  LANGUAGE cpp
  OUT_VAR PROTO_SRCS
  PROTOS ${PROTBUF_FILES}
  IMPORT_DIRS ${SRC_DIR}/protobuf
  PROTOC_OUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/protobuf_build
  )

# Group protobuf output files in their own subfolder within visual studio.
source_group("protobuf" FILES ${PROTO_SRCS})

However, the generated visual studio solution files assume that the output is relative to the CMAKE_CURRENT_SOURCE_DIR which appears to mess things up when the build directory is more than one level down from the actual source code. In the above example here's what is generated from protobuf-config.cmake:

CMAKE_CURRENT_SOURCE_DIR: ${SRCDIR}/cmake_server/DMSO

_rel_dir: ../../protobuf

_abs_dir: ${SRCDIR}/protobuf

_generated_srcs: ${SRCDIR}/cmake_server/DMSO/cmake_build_vs2017/protobuf_build/../../protobuf/pb_hla_utilities.pb.h

                          ${SRCDIR}/cmake_server/DMSO/cmake_build_vs2017/protobuf_build/../../protobuf/pb_hla_utilities.pb.cc

This causes visual studio to create the output directory "protobuf" in

  ${SRCDIR}/cmake_server/DMSO

and then protoc is directed to write the output into

  ${SRCDIR}/cmake_server/DMSO/cmake_build_vs2017/protobuf_build/

which then fails the build.

What I think should be happening is the output files should always be created in PROTOC_OUT_DIR.

I'm not sure what the reason is to not do it this way, but a way to make this work is to add a new option to maintain backward compatability to the protobuf-config.cmake script.

Here's a patch to do that based off of current master:

----------------------------------------------------------------------

diff --git a/cmake/protobuf-config.cmake.in b/cmake/protobuf-config.cmake.in
index 29e39d88..33b1b33f 100644
--- a/cmake/protobuf-config.cmake.in
+++ b/cmake/protobuf-config.cmake.in
@@ -10,7 +10,13 @@ include("${CMAKE_CURRENT_LIST_DIR}/protobuf-targets.cmake")
 function(protobuf_generate)
   include(CMakeParseArguments)
 
-  set(_options APPEND_PATH)
+  # USE_ABSOLUTE_OUTPUT_DIR: If set, create the protobuf output files in the directory pointed to by PROTOC_OUT_DIR.
+  #   If not set, the protobuf output files are created in the directory pointed to by
+  #   PROTOC_OUT_DIR but assumed to be created relative to CMAKE_CURRENT_SOURCE_DIR.
+  # APPEND_PATH: If set, add the path for each file created by protoc as arguments to -I.
+  #   If not set, use CMAKE_CURRENT_SOURCE_DIR as the include path.
+  set(_options APPEND_PATH USE_ABSOLUTE_OUTPUT_DIR)
+ 
   set(_singleargs LANGUAGE OUT_VAR EXPORT_MACRO PROTOC_OUT_DIR)
   if(COMMAND target_sources)
     list(APPEND _singleargs TARGET)
@@ -98,7 +104,11 @@ function(protobuf_generate)
 
     set(_generated_srcs)
     foreach(_ext ${protobuf_generate_GENERATE_EXTENSIONS})
-      list(APPEND _generated_srcs "${protobuf_generate_PROTOC_OUT_DIR}/${_rel_dir}/${_basename}${_ext}")
+      if (${protobuf_generate_USE_ABSOLUTE_OUTPUT_DIR})
+        list(APPEND _generated_srcs "${protobuf_generate_PROTOC_OUT_DIR}/${_basename}${_ext}")
+      else()
+        list(APPEND _generated_srcs "${protobuf_generate_PROTOC_OUT_DIR}/${_rel_dir}/${_basename}${_ext}")
+      endif()
     endforeach()
     list(APPEND _generated_srcs_all ${_generated_srcs})
 

[Chase Bradford]

cmake really needs to change the output file guessing logic to be consistent with protoc.  Specifically, it tracks the list of paths that will be added to the protoc search path, so for every .proto file argument, it should compare prefixes against the search paths in order until the first one is found, then extract the relative path from that point.

For example, given the macro calls

protobuf_generate_cpp(SOURCES, HEADERS, proto/app.proto proto/lib/lib.proto)
SOURCES == app.pb.cc lib.pb.cc

but the protoc command for lib.proto is

protoc -I proto -I proto/lib proto/lib/lib.proto

which generates lib/lib.pb.cc

If the two files are reversed, then the "guessed" output files will match the generated, but only because -I proto and -I proto/lib are reversed.

Even if you disable the automatic appending of paths, CMAKE_CURRENT_SOURCE_DIR is still prepended to the protoc search path, which can make getting exactly what you want difficult.

I had to replace the protobuf_generate macro logic and avoid using the built in module to gain enough control over directory handling to make .proto libraries with full directory structures.  The end result didn't automatically add anything to the protoc search paths and only used what I listed in Protobuf_IMPORT_DIRS.  Likewise, the generated paths were based on search my input list of import dirs in order, finding the first prefix match, and generating the output file by stripping the matching prefix and appending to the remaining relative path to the target output directory.

Regards,

Chase