missing symbols from static CMake build

[Michał Frątczak]

Hello group,

I pulled latest repo ( Jan 4, 2018 ) and performed static build with provided CMake scripts.

I successfully received libopenvdb.a and now I try to link it to my maya plugin, but getting missing symbol INVALID_IDX

// Error: line 1: Unable to dynamically load : /home/mfratczak/dev/vdb_play/levelset/build/Maya/levelset.so

/home/mfratczak/dev/vdb_play/levelset/build/Maya/levelset.so: undefined symbol: _ZN7openvdb4v5_04util11INVALID_IDXE

 Here I found a little cpp program to test DSO loading and it fails on my plugin with the same missing symbol:

./Maya/levelset.so: undefined symbol: _ZN7openvdb4v5_04util11INVALID_IDXE

Checking with nm I get: 

nm /v/pkgs/openvdb/dep/5_0_0/lib/libopenvdb.a | grep INVALID_IDX

0000000000000228 R _ZN7openvdb8v5_1abi34util11INVALID_IDXE

which hints that the symbol is there (as far as I understand nm output)

So, is CMake an experimental build system for OpenVDB (before there was a Makefile only, right?), or I'm doing it wrong ?

I didn't find much problems when building VDB itself, it kind of worked from the start...

I'll be grateful for any tips...

thanks

-michal

[Dan Bailey]

Hi Michal,

If you run those two symbols you've listed through c++filt, you should get a hint as to what the problem might be:

> c++filt _ZN7openvdb4v5_04util11INVALID_IDXE
openvdb::v5_0::util::INVALID_IDX

> c++filt _ZN7openvdb8v5_1abi34util11INVALID_IDXE
openvdb::v5_1abi3::util::INVALID_IDX

As you can see, the library that you've built is version-namespaced differently to the symbols that the plugin is looking for when you're attempting to link it against the library.

This is most likely because you're not passing the same ABI flags to the compiler when compiling your plugin (and perhaps you've even including different headers as the version is different too).

Dan

[Michael Rochefort]

Interesting, I was running into that a while ago when I tried building the Maya plugin statically. In my CMake file I pretty much pointed everything to static libs and set all dependencies that I could to XXX_USE_STATIC_LIBS=ON. Let’s just say the compiler didn’t like that, particularly OpenEXR/Ilmbase. Boost allowed it through but had a similar error in Maya.

Nice tip Dan, I’ll add that one to the books.

Cheers,
Mike

[Michał Frątczak]

Thanks Dan, this is a valuable info and I'd never get to this for myself. Much appreciated.

I'll now take a look into VDB Cmake and try to understand how it works with respect to name versioning.

Just to add info, here is my CMake line setting config flags. There's really nothing more right now:

add_definitions( -D_BOOL -DLINUX -DREQUIRE_IOSTREAM -fPIC -fno-strict-aliasing -ftemplate-depth-100 )

thanks !

[Michał Frątczak]

Michael, did you finally manage to link your plugins to static VDB ?

I have EXR build statically too and I *think* it doesn't make any problems...

[Michael Rochefort]

Unfortunately, no I was not. Luckily I had a backup version I could revert to because for the life of me I couldn’t get a working build compiled after. Right now I need to include libHalf, libboost_system, and libboost_iostreams as part of the LD_LIBRARY_PATH before launching Maya (didn’t want them to be in the Maya install directory. Used to need libblosc, but was able to fix that one.

This makes me want to try again ;)

Cheers,
Mike

[Michał Frątczak]

Reporting back

I was able to fix ABI errors by just disabling CMake option "DOPENVDB_ENABLE_3_ABI_COMPATIBLE"

I also had some success building static openvdb by adding CMake option SET ( CMAKE_POSITION_INDEPENDENT_CODE ON ) to my EXR static builds (both IlmBas and IlmImf). I'm not sure what longterm effects it has, but for now I'm able to load my maya plugin without any LD_LIBRARY_PATH links to exr or vdb so files.

To be continued after weekend...

thanks 

-michal

[Michael Rochefort]

Here's a simple longterm effect (not sure if it'll matter to you): most popular render engines are build with ABI 3 compatibility. Redshift and RenderMan both use 3.1.0, Arnold I think uses 4.0.2 with 3 compatibility, and Houdini uses 3.3.0, which is a customized 4.x build with 3 compatibility. All using ABI3, so if you want to be able to render in a DCC package with a non proprietary engine, you might want that flag back on ;)

In better news, I've taken another stab at getting the OpenVDB Maya plugin statically setup. Even though it comes out as a shared object, it looks like if the standard libopenvdb.* was scripted with static libraries in mind it'll carry over. For example, I used to need libboost_iostreams.so.1.61.0, libboost_system.so.1.61.0, and libHalf.so.23. I just rebuilt and now I don't need libHalf! Here's my build script: https://pastebin.com/WeEUmBVC

If I try adding in "-D Boost_USE_STATIC_LIBS=ON \", the build fails pretty early on:

/usr/bin/ld: /home/mroche/sw/vfx/boost/1.61/lib/libboost_iostreams.a(file_descriptor.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/home/mroche/sw/vfx/boost/1.61/lib/libboost_iostreams.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [openvdb/libopenvdb.so.4.0.2] Error 1
make[1]: *** [openvdb/CMakeFiles/openvdb_shared.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Why this error occurs, I don't know considering I export CFLAGS="-fPIC" and CXXFLAGS="-fPIC" before I start building any of the libraries, so that configuration should have been included. So for now I still need those two boost libraries in LD_LIBRARY_PATH while Maya loads.

Cheers,
Mike

[Michał Frątczak]

Hi Michael and thanks for ABI tips, I'll try to test some more this week (against 3delight in my case, I will also try arnold).

As to linker relocation errors, I was able to fix them with CMAKE option: