Difficulty building copperhead from sources

[Richard Gomes]

Hello,

I've decided to build Python, NumPy, etc and Copperhead from sources.
The difficulty I'm facing now is that libcopperhead.so cannot find the correct location of libcudart.so.5.0 :

(py27)rgomes@terra:~/sources/copperhead$ ldd backend/build/libcopperhead.so
    linux-vdso.so.1 =>  (0x00007fff751ff000)
    libcudart.so.5.0 => not found
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f4f5130e000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f4f510f7000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4f50d54000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4f50ad0000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f4f519e6000)

In my environment, I've installed CUDA toolkit under /opt/local/cuda.
Then I've created a symbolic link like this: /usr/local/cuda --> /opt/local/cuda

(py27)rgomes@terra:~/sources/copperhead$ ll /usr/local/cuda/lib64/libcudart.so*
lrwxrwxrwx 1 rgomes rgomes     16 2012-12-05 02:22 /usr/local/cuda/lib64/libcudart.so -> libcudart.so.5.0*
lrwxrwxrwx 1 rgomes rgomes     19 2012-12-05 02:22 /usr/local/cuda/lib64/libcudart.so.5.0 -> libcudart.so.5.0.35*
-rwxr-xr-x 1 rgomes rgomes 367560 2012-12-05 02:22 /usr/local/cuda/lib64/libcudart.so.5.0.35*

In logs I can see how libcopperhead.so is linked, and apparently everything goes well. No errors at all:

g++ -o backend/build/libcopperhead.so -shared backend/build/allocate.os backend/build/backend_translate.os backend/build/compiler.os backend/build/containerize.os backend/build/cpp_printer.os backend/build/cppnode.os backend/build/ctype.os backend/build/dereference.os backend/build/expression.os backend/build/find_includes.os backend/build/functorize.os backend/build/iterizer.os backend/build/monotype.os backend/build/node.os backend/build/phase_analyze.os backend/build/polytype.os backend/build/prune.os backend/build/py_printer.os backend/build/repr_printer.os backend/build/statement.os backend/build/tuple_break.os backend/build/type.os backend/build/type_convert.os backend/build/type_printer.os backend/build/typedefify.os backend/build/wrap.os backend/build/builtins/decl.os backend/build/builtins/phase.os backend/build/import/paths.os backend/build/thrust/decl.os backend/build/thrust/rewrites.os backend/build/utility/container_type.os backend/build/utility/markers.os backend/build/utility/name_supply.os backend/build/utility/snippets.os backend/build/prelude/runtime/chunk.os backend/build/prelude/runtime/cuarray.os backend/build/prelude/runtime/make_type_holder.os backend/build/prelude/runtime/mempool.os backend/build/prelude/runtime/tags.os -L/usr/local/cuda/lib64 -L/opt/local/lib -L/opt/local/lib/python2.7/config -L/opt/local/lib -lpython2.7 -lcudart

I've already tried to push the linker to find the library, like this:
      $ export LD_RUN_PATH=/usr/local/cuda/lib64

.. but without luck :(

Could someone enlighten me why libcudart.so is not found?

Thanks a lot :)


==============================

For your reference, I'm building python, numpy, etc... copperhead like this:

#!/bin/bash -x

export PREFIX=/opt/local



function create_opt_local() {
rm -r -f ~/.virtualenvs
rm -r -f ${PREFIX}/bin ${PREFIX}/include ${PREFIX}/lib ${PREFIX}/share
sudo mkdir -p ${PREFIX}/bin ${PREFIX}/include ${PREFIX}/lib ${PREFIX}/share
sudo chown -R rgomes:rgomes ${PREFIX}
}


#--------------------------------------------------
# essential tools
#--------------------------------------------------
function install_essential_tools() {
sudo apt-get install zip unzip bzip2 gzip xz-utils wget curl -y
sudo apt-get install build-essential gcc g++ gfortran -y
sudo apt-get install autoconf scons pkg-config -y
sudo apt-get install git bzr mercurial -y
}


#--------------------------------------------------
# downloading everything
#--------------------------------------------------
function download_everything() {
mkdir -p ~/Downloads
pushd ~/Downloads
wget http://sourceforge.net/projects/boost/files/boost/1.48.0/boost_1_48_0.tar.bz2
wget http://thrust.googlecode.com/files/thrust-1.6.0.zip
wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.xz
wget http://downloads.sourceforge.net/project/numpy/NumPy/1.6.2/numpy-1.6.2.tar.gz
wget http://downloads.sourceforge.net/project/scipy/scipy/0.11.0/scipy-0.11.0.tar.gz
wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.10.tar.bz2
wget http://www.hdfgroup.org/ftp/lib-external/szip/2.1/src/szip-2.1.tar.gz
wget http://downloads.sourceforge.net/project/pytables/pytables/2.4.0/tables-2.4.0.tar.gz
wget http://vitables.googlecode.com/files/ViTables-2.1.tar.gz
wget http://pypi.python.org/packages/source/p/pandas/pandas-0.9.1.tar.gz
popd
}


#--------------------------------------------------
# build Python from sources
#--------------------------------------------------


# [01:05] <MindDrive> frgomes, just in case you're morbidly curious, I keep notes on all my own builds, here's
# the configure I used for Python 2.7.3 on my Solaris 10 system:
# CPPFLAGS="-I/software/include -I/usr/sfw/include" \
# LDFLAGS="-L/software/lib -R/software/lib -L/usr/sfw/lib -R/usr/sfw/lib" \
# ./configure --enable-shared --prefix=/software/python-2.7.3


function build_python() {
cd ~/sources
sudo apt-get install libicu44 libicu-dev zlib1g-dev libbz2-dev libncurses5-dev libreadline-gplv2-dev libsqlite3-dev libssl-dev libgdbm-dev -y
#--
tar xf ~/Downloads/Python-2.7.3.tar.xz
cd Python-2.7.3
export CC=gcc
export CXX=g++
export LD_RUN_PATH=${PREFIX}/lib
./configure --prefix=${PREFIX} --enable-shared --enable-unicode=ucs4 --with-pydebug
make -j 8
#make test
make install
unset CC CXX LD_RUN_PATH
}


#--------------------------------------------------
# install package managers
#--------------------------------------------------
function install_package_managers() {
cd ~/sources
curl http://python-distribute.org/distribute_setup.py          | ${PREFIX}/bin/python
curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | ${PREFIX}/bin/python
}


#--------------------------------------------------
# install virtualenv and virtualenvwrapper
#--------------------------------------------------
function install_virtualenv() {
cd ~/sources
sudo apt-get install python-virtualenv virtualenvwrapper -y
}



#--------------------------------------------------
# virtual environment for Python 2.7
#--------------------------------------------------
function create_virtualenv_py27() {
mkvirtualenv --no-site-packages --python=${PREFIX}/bin/python py27
#--
echo "export PATH=${PREFIX}/bin:${PREFIX}/cuda/bin:$PATH"  >> ~/.virtualenvs/py27/bin/postactivate
echo "export LD_LIBRARY_PATH=${PREFIX}/lib"                >> ~/.virtualenvs/py27/bin/postactivate
}


#--------------------------------------------------
# install other packages with pip
#--------------------------------------------------
function install_packages_1() {
pip install setuptools --upgrade
pip install distribute --upgrade
pip install Sphinx
pip install Cython
pip install tornado
pip install pyzmq
pip install ipython
pip install uncertainties
}


#--------------------------------------------------
# build numpy from sources
#--------------------------------------------------
function build_numpy() {
cd ~/sources
sudo apt-get install libatlas-base-dev libblas-dev libatlas-base-dev -y
tar xf ~/Downloads/numpy-1.6.2.tar.gz
cd numpy-1.6.2
rm -r -f build
python --version
python setup.py --requires
python setup.py build
python setup.py install
}


#--------------------------------------------------
# build scipy from sources
#--------------------------------------------------
function build_scipy() {
cd ~/sources
tar xf ~/Downloads/scipy-0.11.0.tar.gz
cd scipy-0.11.0
rm -r -f build
python --version
python setup.py --requires
python setup.py build
python setup.py install
}



#--------------------------------------------------
# install other packages with pip
#--------------------------------------------------
function install_packages_2() {
pip install numexpr
}


#--------------------------------------------------
# build szip from sources
#--------------------------------------------------
function build_szip() {
cd ~/sources
sudo apt-get install zlib-bin -y
#--
tar xf ~/Downloads/szip-2.1.tar.gz
cd szip-2.1
./configure --prefix=${PREFIX} --enable-encoding
make clean
make -j 8
make install
}


#--------------------------------------------------
# build HDF5 from sources - requires szip
#--------------------------------------------------
function build_hdf5() {
cd ~/sources
sudo apt-get build-dep hdf5 -y
tar xf ~/Downloads/hdf5-1.8.10.tar.bz2
cd hdf5-1.8.10
CC=/usr/bin/mpicc ./configure --prefix=${PREFIX} --enable-shared --enable-parallel --with-szlib=${PREFIX}
make clean
make -j 8
make check
make install
make check-install
make clean
}


#--------------------------------------------------
# build pytables from sources
#--------------------------------------------------
function build_pytables() {
cd ~/sources
sudo apt-get install libbz2-dev liblzo2-dev -y
#--
tar xf ~/Downloads/tables-2.4.0.tar.gz
cd tables-2.4.0/
python --version
python setup.py --requires
CC=/usr/bin/mpicc HDF5_DIR=${PREFIX} python setup.py install
}


#--------------------------------------------------
# build vitables from sources
#--------------------------------------------------
function build_vitables() {
cd ~/sources
tar xf ~/Downloads/ViTables-2.1.tar.gz
cd ViTables-2.1/
python --version
python setup.py --requires
python setup.py build
python setup.py install
}


#--------------------------------------------------
# build pandas from sources
#--------------------------------------------------
function build_pandas() {
cd ~/sources
tar xf ~/Downloads/pandas-0.9.1.tar.gz
cd pandas-0.9.1
rm -r -f build dist
python --version
python setup.py --requires
python setup.py build
python setup.py install
}


#--------------------------------------------------
# build boost from sources
#--------------------------------------------------
function build_boost() {
cd ~/sources
tar xf ~/Downloads/boost_1_48_0.tar.bz2
cd boost_1_48_0
#-- install on ${PREFIX}
mkdir -p ${PREFIX}
#--
./bootstrap.sh --prefix=${PREFIX} --with-python=${PREFIX}/bin/python --with-icu
./b2 -j 8
./b2 install
}


#--------------------------------------------------
# install thrust library
#--------------------------------------------------
function build_thrust() {
pushd ${PREFIX}/include
unzip ~/Downloads/thrust-1.6.0.zip
popd
}


#--------------------------------------------------
# build copperhead from sources
#--------------------------------------------------
function build_copperhead() {
cd ~/sources
git clone http://github.com/copperhead/copperhead.git
cd copperhead
#--
rm -r -f dist stage .sconf_temp .sconsign.dblite config.log
rm -r -f ${PREFIX}/lib/python2.7/site-packages/copperhead*
#--
sudo rm /usr/local/cuda
sudo ln -s ${PREFIX}/cuda /usr/local/cuda

#--
cat << EOD > siteconf.py
BOOST_INC_DIR = "${PREFIX}/include"
BOOST_LIB_DIR = "${PREFIX}/lib"
BOOST_PYTHON_LIBNAME = "boost_python"
CUDA_INC_DIR = "/usr/local/cuda/include"
CUDA_LIB_DIR = "/usr/local/cuda/lib64"
NP_INC_DIR = "${PREFIX}/lib/python2.7/site-packages/numpy/core/include"
TBB_INC_DIR = None
TBB_LIB_DIR = None
THRUST_DIR = "${PREFIX}/include"
EOD
cat siteconf.py
#--
python --version
python setup.py install
#--
##unset LD_LIBRARY_PATH
}



#--------------------------------------------------
# main script
#--------------------------------------------------

create_opt_local
install_essential_tools

download_everything

build_python
read -p "Press any key to continue..."

install_package_managers
read -p "Press any key to continue..."

install_virtualenv
source /etc/bash_completion.d/virtualenvwrapper
read -p "Press any key to continue..."

create_virtualenv_py27
read -p "Press any key to continue..."

workon py27
read -p "Press any key to continue..."

install_packages_1
read -p "Press any key to continue..."

build_numpy
read -p "Press any key to continue..."

build_scipy
read -p "Press any key to continue..."

install_packages_2
read -p "Press any key to continue..."

build_szip
read -p "Press any key to continue..."

build_hdf5
read -p "Press any key to continue..."

build_pytables
read -p "Press any key to continue..."

build_vitables
read -p "Press any key to continue..."

build_pandas
read -p "Press any key to continue..."

build_boost
read -p "Press any key to continue..."

build_thrust
read -p "Press any key to continue..."

build_copperhead
read -p "Press any key to continue..."

echo "done."

[Richard Gomes]

OK. I did

    $ sudo ldconfig /opt/local/cuda/lib64

and now libcopperhead.so contains all references OK.

Just add this line just before building Copperhead.


Thanks

Richard Gomes

[Bryan Catanzaro]

Glad to hear you figured it out, thanks for the report!

- bryan

[Richard Gomes]

Just created a blog entry for this:

http://neo-rgomes.blogspot.co.uk/2012/12/python-stack-for-gpu-programming.html

Thanks

--Richard

[Richard Gomes]

The blog entry is here: http://rgomes-info.blogspot.com/2013/01/python-stack-for-gpu-programming.html

-- Richard Gomes

[Bryan Catanzaro]

Thanks, Richard!

- bryan