TOOL_BINS build fails with "undefined reference to `lzma_index_size@XZ_5.0'

[Krishnendu Chaudhury]

Hi All,

  My caffe build progresses happily thru building the libcaffe.so but then when it tries to build the TOOL_BINS stuff (e.g., convert_imageset.bin etc) it fails. Sample build results are appended at the end. 

The problem started when I moved to python 3.4 on anaconda3, before this I was getting the caffe runtests to pass

Essentially, the next line illustrates the problem

/usr/lib/x86_64-linux-gnu/libunwind.so.8: undefined reference to `lzma_index_size@XZ_5.0'

I tried to debug a bit. The undefined symbols (e.g. lzma_index_size) are referenced by libunwind.so and they are defined in liblzma.so (I verified this as follows

$ nm -D -C -g /usr/local/anaconda3/lib/liblzma.so.5 | grep "lzma_index_size"

   0000000000003ef0 T lzma_index_size

)

and liblzma.so is indeed in the linker's path (I verified this as follows

$ ldconfig -p | grep liblzma.so

liblzma.so.5 (libc6,x86-64) => /usr/local/anaconda3/lib/liblzma.so.5
liblzma.so.5 (libc6,x86-64) => /lib/x86_64-linux-gnu/liblzma.so.5
liblzma.so.5 (libc6) => /lib/i386-linux-gnu/liblzma.so.5

liblzma.so (libc6,x86-64) => /usr/local/anaconda3/lib/liblzma.so

).

Thus liblzma.so should be linked in and it does define the problem symbols, yes? What am I missing here?

Thanks,

   Krish

<-----------------begin build result ------------------------->

krishnendu@krishnendu-Leopard-WS:/usr/local/src/caffe$ sudo make all

bc: symbol lookup error: /usr/local/anaconda3/lib/libreadline.so.6: undefined symbol: PC

CXX/LD -llzma -o .build_release/tools/convert_imageset.bin

/usr/lib/x86_64-linux-gnu/libunwind.so.8: undefined reference to `lzma_index_size@XZ_5.0'

/usr/lib/x86_64-linux-gnu/libunwind.so.8: undefined reference to `lzma_stream_footer_decode@XZ_5.0'

/usr/lib/x86_64-linux-gnu/libunwind.so.8: undefined reference to `lzma_index_uncompressed_size@XZ_5.0'

/usr/lib/x86_64-linux-gnu/libunwind.so.8: undefined reference to `lzma_stream_buffer_decode@XZ_5.0'

/usr/lib/x86_64-linux-gnu/libunwind.so.8: undefined reference to `lzma_index_buffer_decode@XZ_5.0'

/usr/lib/x86_64-linux-gnu/libunwind.so.8: undefined reference to `lzma_index_end@XZ_5.0'

collect2: error: ld returned 1 exit status

Makefile:559: recipe for target '.build_release/tools/convert_imageset.bin' failed

make: *** [.build_release/tools/convert_imageset.bin] Error 1

[Krishnendu Chaudhury]

Also here is some more info:

$ nm -D -C /usr/lib/x86_64-linux-gnu/libunwind.so.8 | grep lzma

                 U lzma_index_buffer_decode

                 U lzma_index_end

                 U lzma_index_size

                 U lzma_index_uncompressed_size

                 U lzma_stream_buffer_decode

                 U lzma_stream_footer_decode

[Krishnendu Chaudhury]

I think what is happening here is libunwind.so (part of my linux distro) dynamically links liblzma.so. Now when I installed anaconda my shared object search path got modified so that the anaconda version of liblzma.so comes in front of the older (system provided) liblzma.so. It seems the linker refuses to link in a .so whose version is newer than the one with which the original binary (libunwind.so) was produced. Can anybody verify this statement? Also, what is a workaround?

[Krishnendu Chaudhury]

OK, problem solved. Here is what was happening: I added hdf5.conf under /etc/ld.conf.so.d which had the line /usr/local/anaconda3/lib. This was pointing caffe build (correctly) to the libhdf5*.so sitting under anaconda. Unfortunately,  /usr/local/anaconda3/lib has a liblzma.so which then overrides the system provided liblzma.so sitting in/lib/x86_64-linux-gnu/. My solution was to delete the hdf5.conf and manually do "ln -s /usr/local/anaconda3/lib/libhdf5.so /usr/local/lib" and similarly for libhdf5_hi etc.

[sunpe...@gmail.com]

Hi,I through this way to install hdf5--"sudo apt-get install libhdf5-serial-dev" . But I also have a same error as you.Could you tell me how to solve it?Thank you very much.