Generated source build issues (C++)

[Paul Nienaber (CW)]

Hi,

Joined on another project that's heavily using Protobuf (Apache Arrow) and running into strange macro-and-other-symbol visibility issues when I'm including generated Protobuf headers in more source files (e.g. in application headers).  I'm guessing this may be a more common general problem but I didn't find anything similar enough to the issues I'm encountering when searching for similar errors.  For example I'm seeing (truncated as things get less interesting and somewhat repetitive down the log):

/opt/homebrew/bin/ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DARROW_EXTRA_ERROR_CONTEXT -DARROW_FLIGHT_SQL_EXPORTING -DARROW_HAVE_NEON -DARROW_WITH_RE2 -DARROW_WITH_TIMING_TESTS -DARROW_WITH_ZLIB -DGRPC_NAMESPACE_FOR_TLS_CREDENTIALS_OPTIONS=grpc::experimental -DGRPC_USE_CERTIFICATE_VERIFIER -DGRPC_USE_TLS_CHANNEL_CREDENTIALS_OPTIONS -DURI_STATIC_BUILD -I/Users/pauln/bq/git/arrow/cpp/session-managemen-build-debug/src -I/Users/pauln/bq/git/arrow/cpp/src -I/Users/pauln/bq/git/arrow/cpp/src/generated -isystem /opt/homebrew/include -isystem /Users/pauln/bq/git/arrow/cpp/thirdparty/flatbuffers/include -isystem /Users/pauln/bq/git/arrow/cpp/thirdparty/hadoop/include -isystem /opt/homebrew/opt/openssl@1.1/include -isystem /Users/pauln/bq/git/arrow/cpp/session-managemen-build-debug/jemalloc_ep-prefix/src -fno-aligned-new -Qunused-arguments -fcolor-diagnostics -Wall -Wextra -Wdocumentation -Wshorten-64-to-32 -Wno-missing-braces -Wno-unused-parameter -Wno-constant-logical-operand -Wno-return-stack-address -Wno-unknown-warning-option -Wno-pass-failed -march=armv8-a -g -Werror -O0 -ggdb -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk -fPIC -std=c++17 -MD -MT src/arrow/flight/sql/CMakeFiles/arrow_flight_sql_objlib.dir/client.cc.o -MF src/arrow/flight/sql/CMakeFiles/arrow_flight_sql_objlib.dir/client.cc.o.d -o src/arrow/flight/sql/CMakeFiles/arrow_flight_sql_objlib.dir/client.cc.o -c /Users/pauln/bq/git/arrow/cpp/src/arrow/flight/sql/client.cc

In file included from /Users/pauln/bq/git/arrow/cpp/src/arrow/flight/sql/client.cc:22: 

In file included from /Users/pauln/bq/git/arrow/cpp/src/arrow/flight/sql/client.h:24: 

/Users/pauln/bq/git/arrow/cpp/session-managemen-build-debug/src/arrow/flight/sql/FlightSql.pb.h:47:3: error: expected expression 

static const uint32_t offsets[]; 

^ 

/Users/pauln/bq/git/arrow/cpp/session-managemen-build-debug/src/arrow/flight/sql/FlightSql.pb.h:46:32: error: variable has incomplete type 'struct ARROW_FLIGHT_SQL_EXPORT' 

struct ARROW_FLIGHT_SQL_EXPORT TableStruct_FlightSql_2eproto { 

^ 

/Users/pauln/bq/git/arrow/cpp/session-managemen-build-debug/src/arrow/flight/sql/FlightSql.pb.h:46:8: note: forward declaration of 'ARROW_FLIGHT_SQL_EXPORT' 

struct ARROW_FLIGHT_SQL_EXPORT TableStruct_FlightSql_2eproto { 

^ 

/Users/pauln/bq/git/arrow/cpp/session-managemen-build-debug/src/arrow/flight/sql/FlightSql.pb.h:49:73: error: no member named 'DescriptorTable' in namespace 'google::protobuf::internal' 

ARROW_FLIGHT_SQL_EXPORT extern const ::PROTOBUF_NAMESPACE_ID::internal::DescriptorTable descriptor_table_FlightSql_2eproto; 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ 

/Users/pauln/bq/git/arrow/cpp/session-managemen-build-debug/src/arrow/flight/sql/FlightSql.pb.h:49:88: error: expected ';' after top level declarator 

ARROW_FLIGHT_SQL_EXPORT extern const ::PROTOBUF_NAMESPACE_ID::internal::DescriptorTable descriptor_table_FlightSql_2eproto; 

[...]

Thanks in advance for any insight!

Cheers

~Paul

[Michael Ngarimu]

I’m another protobuf user using protobuf in a much smaller project on Windows. The *_EXPORT macros are present in generated source for using dynamic shared object symbol export annotations. In my experience non-Windows platforms tend to make all symbols public/exported, whereas Windows symbols are exported explicitly. 

It looks like you might be working on macOS. It’s been a while since I’ve developed shared objects on macOS, never worked with protobuf on macOS, and have no familiarity with Arrow Flight builds so maybe others will be able to help directly. Happy to offer further suggestions/help but there’s probably someone with more direct knowledge of Arrow Flight that is aware of the specifics of that project. Naive question, have you checked with them regarding the specifics of their build process on your platform?