Linking error on Fedora 17+ with Sigar -- "undefined reference to sigar_version_get"

[Denis Arnaud]

Hi,

when trying to compile Hypertable on Fedora 17+ (Fedora 17 and Fedora Rawhide), directly from the Git clone:

 $ mkdir -p ~/dev/hypertable

 $ cd ~/dev/hypertable

 $ git clone git://github.com/hypertable/hypertable.git hypertablegit

 $ rm -rf build && mkdir build

 $ cd build

 $ cmake ..

The following Sigar-related linking error appears:

 $ make VERBOSE=1

Linking CXX executable bloom_filter_test

cd /home/build/dev/hypertable/hypertablegit/build/src/cc/Common && /usr/bin/cmake -E cmake_link_script CMakeFiles/bloom_filter_test.dir/link.txt --verbose=1

/usr/lib/ccache/c++   -g    CMakeFiles/bloom_filter_test.dir/tests/bloom_filter_test.cc.o  -o bloom_filter_test -rdynamic libHyperCommon.a -lboost_iostreams -lboost_program_options -lboost_filesystem -lboost_thread-mt -lboost_system -lboost_chrono -lboost_python -lpython2.7 -llog4cpp -lz -lsnappy -lsigar -ldl -lncurses -lpthread -lre2 

libHyperCommon.a(SystemInfo.cc.o): In function `Hypertable::system_info_lib_version()':

/home/build/dev/hypertable/hypertablegit/src/cc/Common/SystemInfo.cc:768: undefined reference to `sigar_version_get'

libHyperCommon.a(SystemInfo.cc.o): In function `Hypertable::system_info_lib_version(std::ostream&)':

/home/build/dev/hypertable/hypertablegit/src/cc/Common/SystemInfo.cc:773: undefined reference to `sigar_version_get'

collect2: error: ld returned 1 exit status

make[2]: *** [src/cc/Common/bloom_filter_test] Error 1

make[2]: Leaving directory `/home/build/dev/hypertable/hypertablegit/build'

make[1]: *** [src/cc/Common/CMakeFiles/bloom_filter_test.dir/all] Error 2

make[1]: Leaving directory `/home/build/dev/hypertable/hypertablegit/build'

make: *** [all] Error 2

-----------------------------------------

Note, for those interested in reproducing it, I have installed the following dependencies (as root):

 yum -y install ant ant-contrib ant-scripts ant-commons-net ant-commons-logging \

  boost-devel ceph-devel ceph-libs ceph-libcephfs ceph-fuse cronolog \

  gperftools-devel gperftools-libs jemalloc-devel hoard \

  java-1.7.0-openjdk-devel libdb-devel libdb-cxx-devel libdb-utils \

  libevent-devel log4cpp-devel perl-Class-Accessor \

  rrdtool-devel rrdtool-ruby ruby-RRDtool sigar-devel snappy-devel

 debuginfo-install -y boost ceph cronolog db gperftools hoard java-1.7.0-openjdk \

  jemalloc libevent log4cpp rrdtool ruby-RRDtool sigar snappy

[Christoph Rupp]

Hi Denis,

i remember that sigar was a bit painfull to install...

if you download the sigar libraries, then the actual library for 64bit linux is named libsigar-amd64-linux.so. But your Makefile links against -lsigar, therefore maybe it doesn't find the library. You could test this by renaming your /usr/local/lib/libsigar-amd64-linux.so to /usr/local/lib/libsigar.so.

You can also look into the CMakeCache.txt in your build directory; it should list the full sigar library path in the dependencies.

If the linker is not able to find the file then one of the cmake scripts has an error.

Thanks
Christoph

2012/10/21 Denis Arnaud <denis....@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Hypertable Development" group.
To view this discussion on the web visit https://groups.google.com/d/msg/hypertable-dev/-/YnSCrIRaWq8J.
To post to this group, send email to hyperta...@googlegroups.com.
To unsubscribe from this group, send email to hypertable-de...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.

[Doug Judd]

Hi Denis,

The SIGAR library isn't available in a yum repository (last we checked).  We usually download it and install it "by hand".  Take a look at our CentOS 5 build setup instructions for how to install SIGAR.  We're happy to host a yum repository for SIGAR, if that's necessary for Fedora.

One thing we've done to reduce the dependencies with our RPM package is to include all of the dependent libraries in the installation.  

[doug@test00 ~]$ ls /opt/hypertable/doug/current/lib/

java/    libHyperThirdParty.a

libboost_filesystem.so.1.44.0    libHyperThrift.a

libboost_iostreams.so.1.44.0    libHyperThriftConfig.a

libboost_program_options.so.1.44.0  libHyperTools.a

libboost_python.so.1.44.0    libk5crypto.so.3

libboost_system.so.1.44.0    libkrb5.so.3

libboost_thread.so.1.44.0    libkrb5support.so.0

libbz2.so.1    liblog4cpp.so.4

libcom_err.so.2    libncurses.so.5

libcrypto.so.10    libpython2.6.so.1.0

libdb_cxx-4.8.so    libre2.so.0

libedit.so.0    libsigar-amd64-linux.so

libevent-1.4.so.2    libsnappy.so.1

libexpat.so.1    libssl.so.10

libgcc_s.so.1    libstdc++.so.6

libgssapi_krb5.so.2    libtcmalloc_minimal.so.4

libHyperAppHelper.a    libtcmalloc.so.4

libHyperComm.a    libthrift-0.8.0.so

libHyperCommon.a    libthriftnb-0.8.0.so

libHyperDfsBroker.a    libunwind.so.7

libHyperDfsCmds.a    libz.so.1

libHyperMaster.a    perl/

libHyperRanger.a    php/

libHyperspace.a    py/

libHypertable.a    rb/

This makes it easy to install the package without having to worry about dependencies.  Is this something that is ok with Fedora?  Also, there are a lot of Java .jar files that we include in the package (Hadoop, thrift, etc.).  If we're allowed to include .jar files in the RPM, then the only java-related dependencies would be Java itself and Hadoop.

- Doug

--
You received this message because you are subscribed to the Google Groups "Hypertable Development" group.
To view this discussion on the web visit https://groups.google.com/d/msg/hypertable-dev/-/YnSCrIRaWq8J.
To post to this group, send email to hyperta...@googlegroups.com.
To unsubscribe from this group, send email to hypertable-de...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.

--

Doug Judd

CEO, Hypertable Inc.

[Nate]

I posted on the same issue. I tried the centos5 build script.. getting a 404 on sigar. How many active developers does hypertable have?

im fairly certain its required to package separately.

[Christoph Rupp]

Hi Nate,

this script is outdated and not really maintained. I remember that i had issues with libsigar when i installed it (on Ubuntu) but i don't really recall what it was. Let me try to reproduce it and i'll send you the build instructions.

bye
Christoph

2012/11/1 Nate <nate....@gmail.com>

To view this discussion on the web visit https://groups.google.com/d/msg/hypertable-dev/-/u4FycGUyH1YJ.

[Denis Arnaud]

Hi Doug,

On Monday, 22 October 2012 17:38:50 UTC+2, Doug Judd wrote:

The SIGAR library isn't available in a yum repository (last we checked).  We usually download it and install it "by hand".  Take a look at our CentOS 5 build setup instructions for how to install SIGAR.  We're happy to host a yum repository for SIGAR, if that's necessary for Fedora.

In fact, SIGAR is already packaged for Fedora (see https://admin.fedoraproject.org/pkgdb/acls/name/sigar). For instance, you can check on the RedHat mirror for Fedora 17: http://dl.fedoraproject.org/pub/fedora/linux/releases/17/Everything/x86_64/os/Packages/s/sigar-1.6.5-0.6.git58097d9.fc17.x86_64.rpm and http://dl.fedoraproject.org/pub/fedora/linux/releases/17/Everything/x86_64/debug/s/sigar-debuginfo-1.6.5-0.6.git58097d9.fc17.x86_64.rpm (for the object symbols).
You will note that Fedora packagers have shipped a pre-release (1.6.5). However, the incriminated symbol, namely sigar_version_get, cannot be found in that version!
$ nm -a -C /usr/lib/debug/usr/lib/libsigar.so.debug | c++filt | grep sigar_version
[empty]

And the library is not empty:
$ nm -a -C /usr/lib/debug/usr/lib/libsigar.so.debug | c++filt | grep sigar_swap_get
00008d90 T sigar_swap_get
$ nm -a -C /usr/lib/debug/usr/lib/libsigar.so.debug | wc -l
633

So, apparently, either that symbol/API has been removed in the latest version of the SIGAR code, or Fedora packagers have done something wrong... I had a quick look at the SIGAR code, but could not find the source of the issue in the 10 minutes I dedicated to it. If you have some more insights, do not hesitate!

One thing we've done to reduce the dependencies with our RPM package is to include all of the dependent libraries in the installation. [...] This makes it easy to install the package without having to worry about dependencies. Is this something that is ok with Fedora? Also, there are a lot of Java .jar files that we include in the package (Hadoop, thrift, etc.).  If we're allowed to include .jar files in the RPM, then the only java-related dependencies would be Java itself and Hadoop.

No, bundling external projects (or even components or libraries) is forbidden. Everything, including the rationale, is explained on the dedicated Fedora Wiki page: https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries. That of course encompasses Java .jar files.
That is why we have to package every single dependency, as suggested in https://fedoraproject.org/wiki/Hypertable

If some volunteers want to help (co-)maintain some of the packages, please do not hesitate; I will of course support them.

Kind regards

Denis
 

[Christoph Rupp]

I built sigar from head last week to look into those problems. The internal version is 1.6.6. And i have those functions:

objdump -x bindings/java/build/obj/amd64-linux/lib/libsigar-amd64-linux.so | grep sigar_version
0000000000000000 l    df *ABS*    0000000000000000              sigar_version.c
0000000000231840 l     O .data    0000000000000048              sigar_version
00000000000180b0 g     F .text    0000000000000008              sigar_version_get

[chris@wintermute sigar.git (sigar-1.6)]$ nm -a -C bindings/java/build/obj/amd64-linux/lib/libsigar-amd64-linux.so | c++filt | grep sigar_version
0000000000231840 d sigar_version
0000000000000000 a sigar_version.c
00000000000180b0 T sigar_version_get

I downloaded the fedora 1.6.5 and it seems a lot of symbols are missing:

objdump -x libsigar.so.debug | wc  (from fedora)
    620    3475   40200

objdump -x bindings/java/build/obj/amd64-linux/lib/libsigar-amd64-linux.so | wc
    793    4465   63587

2012/11/2 Denis Arnaud <denis....@gmail.com>

--

You received this message because you are subscribed to the Google Groups "Hypertable Development" group.

To view this discussion on the web visit https://groups.google.com/d/msg/hypertable-dev/-/-OCCBIT3QSgJ.

[Denis Arnaud]

Thanks Christoph!

On Monday, 5 November 2012 09:34:13 UTC+1, Christoph Rupp wrote:

I built sigar from head last week to look into those problems. The internal version is 1.6.6. And i have those functions:

[chris@wintermute sigar.git (sigar-1.6)]$ nm -a -C bindings/java/build/obj/amd64-linux/lib/libsigar-amd64-linux.so | c++filt | grep sigar_version
0000000000231840 d sigar_version
0000000000000000 a sigar_version.c
00000000000180b0 T sigar_version_get

I downloaded the fedora 1.6.5 and it seems a lot of symbols are missing:
objdump -x libsigar.so.debug | wc  (from fedora)
    620    3475   40200
objdump -x bindings/java/build/obj/amd64-linux/lib/libsigar-amd64-linux.so | wc
    793    4465   63587

I will try to dig more in the intricacies of the Fedora package (probably not before next week-end, however).

-denis