Cross compiling SoapySDR and modules

555 views

Skip to first unread message

Lopez Antony

unread,

Jun 26, 2018, 5:52:25 AM6/26/18

to Pothos Users

Hello,

My name is Antony and I am trying to develop my own headless spectrum analyser software from HackRF and/or RTL-SDR USB sticks. It’s basically the only SDR hardware I currently have. Spectrum data is retrieved using TCP. I decided to use SoapySDR to benefit from the hardware abstraction it offers. My desktop is running Manjaro. I called my software Rubiapy for now (nothing related to Ruby language).

I am making great progress on my desktop PC version, but now I would like to run it on a raspberry-like ARM based board running Debian Jessy. I would like to keep compiling my software on my computer, so I figured out I had to cross-compile my dependencies which are SoapySDR and the modules I use : SoapyHackRF and SoapyRTLSDR. SoapySDR cross-compiles easily but not the modules.

First try

I used this command to cross compile SoapySDR inside the cloned repo in a build directory:

cmake \
    -D CMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc \
    -D CMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++ \
    -D CMAKE_AR=/usr/bin/arm-linux-gnueabihf-ar \
    -D CMAKE_C_COMPILER_AR=/usr/bin/arm-linux-gnueabihf-gcc-ar \
    -D CMAKE_C_COMPILER_RANLIB=/usr/bin/arm-linux-gnueabihf-gcc-ranlib \
    -D CMAKE_LINKER=/usr/bin/arm-linux-gnueabihf-ld \
    -D CMAKE_NM=/usr/bin/arm-linux-gnueabihf-nm \
    -D CMAKE_OBJCOPY=/usr/bin/arm-linux-gnueabihf-objcopy \
    -D CMAKE_OBJDUMP=/usr/bin/arm-linux-gnueabihf-objdump \
    -D CMAKE_RANLIB=/usr/bin/arm-linux-gnueabihf-ranlib \
    -D CMAKE_STRIP=/usr/bin/arm-linux-gnueabihf-strip \
    ..

I get my .so file for ARM successfully.

For SoapyHackRF using the same command, cmake fails at generating Makefile because cmake can’t find the hackRF library nor its include folder :

-- The CXX compiler identification is GNU 8.1.0
-- Check for working CXX compiler: /usr/bin/arm-linux-gnueabihf-g++
-- Check for working CXX compiler: /usr/bin/arm-linux-gnueabihf-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Build type not specified: defaulting to release.
-- Found PkgConfig: /usr/bin/pkg-config (found version "1.4.2") 
-- Could NOT find LIBHACKRF (missing: LIBHACKRF_LIBRARY LIBHACKRF_INCLUDE_DIR) 
CMake Error at CMakeLists.txt:17 (message):
  HackRF development files not found...

The result is the same for SoapyRTLSDR which can’t find the osmocom’s rtl-sdr library.
It looks like I’ll have to cross-compile every single dependency of those Soapy modules.

Second try

I originally posted this question on SoapySDR’s GitHub repo and moved to this Google group thanks to unixpunk advice. In his answer he also advised me to
cross-compile and install those libraries and dependencies in a staging directory. So I made a Manjaro VM on my computer using VirtualBox to make sure I won’t break anything, just in case.
In this VM, I cloned the repos of :

Next to this repo I created a folder named venv which mirrors UNIX root layout:

.
└─ venv
   ├── bin
   ├── lib
   └── usr
       ├── bin
       ├── etc
       ├── include
       ├── lib
       ├── local
       │   ├── bin
       │   ├── etc
       │   ├── include
       │   ├── lib
       │   ├── man
       │   ├── sbin
       │   ├── share
       │   └── src
       ├── sbin
       ├── share
       └── src

Then for each of the repo, I installed them in the venv “fake UNIX root folder”:

Here are the the cmake/configure commands I used :

# SoapySDR
git clone https://github.com/pothosware/SoapySDR.git && mkdir SoapySDR/build && cd SoapySDR/build
cmake \
    -D CMAKE_INSTALL_PREFIX=/home/a/tmprepo/venv/usr/local \
    \
    -D CMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc \
    -D CMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++ \
    -D CMAKE_AR=/usr/bin/arm-linux-gnueabihf-ar \
    -D CMAKE_C_COMPILER_AR=/usr/bin/arm-linux-gnueabihf-gcc-ar \
    -D CMAKE_C_COMPILER_RANLIB=/usr/bin/arm-linux-gnueabihf-gcc-ranlib \
    -D CMAKE_LINKER=/usr/bin/arm-linux-gnueabihf-ld \
    -D CMAKE_NM=/usr/bin/arm-linux-gnueabihf-nm \
    -D CMAKE_OBJCOPY=/usr/bin/arm-linux-gnueabihf-objcopy \
    -D CMAKE_OBJDUMP=/usr/bin/arm-linux-gnueabihf-objdump \
    -D CMAKE_RANLIB=/usr/bin/arm-linux-gnueabihf-ranlib \
    -D CMAKE_STRIP=/usr/bin/arm-linux-gnueabihf-strip \
    ..
make
sudo make install

# libusb-1.0
git clone https://github.com/libusb/libusb.git && cd libusb
./autogen.sh
./configure \
    CC=/usr/bin/arm-linux-gnueabihf-gcc \
    --host=arm-linux \
    --enable-udev=no \
    --enable-shared \
    --prefix=/home/a/tmprepo/venv/usr/local
make
sudo make install

# rtl-sdr
git clone https://github.com/osmocom/rtl-sdr.git && mkdir rtl-sdr/build && cd rtl-sdr/build
cmake \
    -D CMAKE_LIBRARY_PATH=/home/a/tmprepo/venv/usr/local/lib \
    -D CMAKE_INSTALL_PREFIX=/home/a/tmprepo/venv/usr/local \
    -D INSTALL_UDEV_RULES=ON \
    \
    -D CMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc \
    -D CMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++ \
    -D CMAKE_AR=/usr/bin/arm-linux-gnueabihf-ar \
    -D CMAKE_C_COMPILER_AR=/usr/bin/arm-linux-gnueabihf-gcc-ar \
    -D CMAKE_C_COMPILER_RANLIB=/usr/bin/arm-linux-gnueabihf-gcc-ranlib \
    -D CMAKE_LINKER=/usr/bin/arm-linux-gnueabihf-ld \
    -D CMAKE_NM=/usr/bin/arm-linux-gnueabihf-nm \
    -D CMAKE_OBJCOPY=/usr/bin/arm-linux-gnueabihf-objcopy \
    -D CMAKE_OBJDUMP=/usr/bin/arm-linux-gnueabihf-objdump \
    -D CMAKE_RANLIB=/usr/bin/arm-linux-gnueabihf-ranlib \
    -D CMAKE_STRIP=/usr/bin/arm-linux-gnueabihf-strip \
    ..
make
sudo make install

# hackrf
# Best to comment out `add_subdirectory(hackrf-tools)` in hackrf's CMakeList.txt
# Otherwise hackrf-tools will be compiled and we do not want to also cross compile its dependencies
# We just need libhackrf.so here.
git clone https://github.com/mossmann/hackrf.git && mkdir hackrf/host/build && cd hackrf/host/build
cmake \
    -D CMAKE_LIBRARY_PATH=/home/a/tmprepo/venv/usr/local/lib \
    -D CMAKE_INSTALL_PREFIX=/home/a/tmprepo/venv/usr/local \
    \
    -D CMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc \
    -D CMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++ \
    -D CMAKE_AR=/usr/bin/arm-linux-gnueabihf-ar \
    -D CMAKE_C_COMPILER_AR=/usr/bin/arm-linux-gnueabihf-gcc-ar \
    -D CMAKE_C_COMPILER_RANLIB=/usr/bin/arm-linux-gnueabihf-gcc-ranlib \
    -D CMAKE_LINKER=/usr/bin/arm-linux-gnueabihf-ld \
    -D CMAKE_NM=/usr/bin/arm-linux-gnueabihf-nm \
    -D CMAKE_OBJCOPY=/usr/bin/arm-linux-gnueabihf-objcopy \
    -D CMAKE_OBJDUMP=/usr/bin/arm-linux-gnueabihf-objdump \
    -D CMAKE_RANLIB=/usr/bin/arm-linux-gnueabihf-ranlib \
    -D CMAKE_STRIP=/usr/bin/arm-linux-gnueabihf-strip \
    ..
make
sudo make install

# SoapyHackRF
git clone https://github.com/pothosware/SoapyHackRF.git && mkdir SoapyHackRF/build && cd SoapyHackRF/build
cmake \
    -D CMAKE_LIBRARY_PATH=/home/a/tmprepo/venv/usr/local/lib \
    -D CMAKE_PREFIX_PATH=/home/a/tmprepo/venv/usr/local/share/cmake \
    -D LIBHACKRF_INCLUDE_DIR=/home/a/tmprepo/venv/usr/local/include/libhackrf \
    -D CMAKE_INSTALL_PREFIX=/home/a/tmprepo/venv/usr/local \
    \
    -D CMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc \
    -D CMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++ \
    -D CMAKE_AR=/usr/bin/arm-linux-gnueabihf-ar \
    -D CMAKE_C_COMPILER_AR=/usr/bin/arm-linux-gnueabihf-gcc-ar \
    -D CMAKE_C_COMPILER_RANLIB=/usr/bin/arm-linux-gnueabihf-gcc-ranlib \
    -D CMAKE_LINKER=/usr/bin/arm-linux-gnueabihf-ld \
    -D CMAKE_NM=/usr/bin/arm-linux-gnueabihf-nm \
    -D CMAKE_OBJCOPY=/usr/bin/arm-linux-gnueabihf-objcopy \
    -D CMAKE_OBJDUMP=/usr/bin/arm-linux-gnueabihf-objdump \
    -D CMAKE_RANLIB=/usr/bin/arm-linux-gnueabihf-ranlib \
    -D CMAKE_STRIP=/usr/bin/arm-linux-gnueabihf-strip \
    ..
make
sudo make install

# SoapyRTLSDR
git clone https://github.com/pothosware/SoapyRTLSDR.git && mkdir SoapyRTLSDR/build && cd SoapyRTLSDR/build
cmake \
    -D CMAKE_LIBRARY_PATH=/home/a/tmprepo/venv/usr/local/lib \
    -D CMAKE_PREFIX_PATH=/home/a/tmprepo/venv/usr/local/share/cmake \
    -D CMAKE_INSTALL_PREFIX=/home/a/tmprepo/venv/usr/local \
    \
    -D CMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabihf-gcc \
    -D CMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabihf-g++ \
    -D CMAKE_AR=/usr/bin/arm-linux-gnueabihf-ar \
    -D CMAKE_C_COMPILER_AR=/usr/bin/arm-linux-gnueabihf-gcc-ar \
    -D CMAKE_C_COMPILER_RANLIB=/usr/bin/arm-linux-gnueabihf-gcc-ranlib \
    -D CMAKE_LINKER=/usr/bin/arm-linux-gnueabihf-ld \
    -D CMAKE_NM=/usr/bin/arm-linux-gnueabihf-nm \
    -D CMAKE_OBJCOPY=/usr/bin/arm-linux-gnueabihf-objcopy \
    -D CMAKE_OBJDUMP=/usr/bin/arm-linux-gnueabihf-objdump \
    -D CMAKE_RANLIB=/usr/bin/arm-linux-gnueabihf-ranlib \
    -D CMAKE_STRIP=/usr/bin/arm-linux-gnueabihf-strip \
    ..
make
sudo make install

This gave me all the .so I needed. But of course it’s not over. After copying all the .so and headers onto my host OS next to my Makefile in a directory called ARM, if I try to cross compile my software, I get a new error I have never seen before:

arm-linux-gnueabihf-g++ -Wall -Werror -O2 -D ARM -I ARM/include/ -I ./src/network -I ./src/core -I ./src/command -I ./src/bench  -std=c++11   -L ARM/lib/  _build/Rubiapy/benchmark.o _build/Rubiapy/command.o _build/Rubiapy/HybridServer.o _build/Rubiapy/HybridClient.o _build/Rubiapy/UDPAgent.o _build/Rubiapy/NetworkAgent.o _build/Rubiapy/main.o -lpthread -lSoapySDR -lHackRFSupport -lhackrf -lrtlsdrSupport -lrtlsdr -lusb-1.0 -ludev -lrt -ldl -o _build/Rubiapy.elf
venv/lib/libSoapySDR.so : undefined reference to « glob@GLIBC_2.27 »
collect2: error: ld returned 1 exit status
make: *** [Makefile:180: _build/Rubiapy.elf] Error 1

Googling it helped me understand that a glob function is called in SoapySDR.so and it seems that it causes errors with the glibc I have on my computer. I am out of idea for this one.
The most attentive noticed the -ludev -lrt -ldl. Those are libs I copied from my ARM board.

I hope my explanations are not too messy. Please feel free to ask any other information if I am not clear enough.

Regards

Antony L

Lopez Antony

unread,

Jun 26, 2018, 9:18:00 AM6/26/18

to Pothos Users

Hello, it’s me again,

I also compiled everything on my ARM board then copy the .so files to my desktop. Next I tried to link my software with those copied .so but now I get another error:

arm-linux-gnueabihf-g++ -Wall -Werror -O2 -D ARM -I ARM/include/ -I ./src/network -I ./src/core -I ./src/command -I ./src/bench  -std=c++11   -L ARM/lib -L ARM/lib/base  _build/Rubiapy/benchmark.o _build/Rubiapy/command.o _build/Rubiapy/HybridServer.o _build/Rubiapy/HybridClient.o _build/Rubiapy/UDPAgent.o _build/Rubiapy/NetworkAgent.o _build/Rubiapy/main.o -lpthread -lSoapySDR -lHackRFSupport -lhackrf -lrtlsdrSupport -lrtlsdr -lusb-1.0 -ludev -lrt -ldl -o _build/Rubiapy.elf
_build/Rubiapy/benchmark.o : In the function « findDevices(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) » :
benchmark.cpp:(.text+0x8170) : undefined reference to  « SoapySDR::Device::enumerate(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) »
benchmark.cpp:(.text+0x87c8) : undefined reference to « SoapySDR::KwargsToString(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&) »
collect2: error: ld returned 1 exit status
make: *** [Makefile:181: _build/Rubiapy.elf] Error 1

However if I dump the libSoapySDR.so from my ARM board, I can find the function which are causing the undefined reference :

> nm -gC ARM/lib/libSoapySDR.so  | grep enum
0002bf59 T SoapySDRDevice_enumerate
0002bda5 T SoapySDRDevice_enumerateStrArgs
000108f1 T SoapySDR::Device::enumerate(std::string const&)
00010321 T SoapySDR::Device::enumerate(std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)

> nm -gC ARM/lib/libSoapySDR.so  | grep KwargsToString
0001566d T SoapySDR::KwargsToString(std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)

I don’t understand why I keep getting those undefined reference error even if the functions are present.

Please let me know if yout need more details. I have to admit I am running out of ideas to cross compile my software.

Regards

Antony L

Steve H

unread,

Jun 26, 2018, 9:47:49 AM6/26/18

to Pothos Users

Hi Antony, Steve (unixpunk) here. This looks like progress now that you're able to get all the deps and modules compiled. I wonder if you're going to need a full stage of the Pi's root laid down, then your modules 'installed' over it for this to work. This was the scenario that I've always used. My experience is cross-compiling for the PlutoSDR which is an armv7 NEON (like the Pi1 and 2 I think?) but with this I'm able to get the full staging filesystem out of the firmware build and have always laid that down first and then used the cmake root path option (syntax escapes me) so it pulled all includes/libs from there. (You'd use additional -L options to do this since no cmake on your app) This might be what's missing, there's no glibc in the staging folder for it to use? (Just guessing) I'm not sure what's in the ARM/lib and ARM/include, but maybe these aren't exact/complete with the same glibc version as expected by SoapySDR since it wasn't told to use those same folders during its build process? (Or the libs from the ARM board don't match what's in the ARM folder?) HTH

Lopez Antony

unread,

Jun 27, 2018, 8:58:25 AM6/27/18

to Pothos Users

Hello again,

Thank you Steve for your answer.

So following your ideas, I made quite a lot of tries with all those .so on my desktop and my VMs. Aaaannnndd as always, part of the problem was between the chair and the screen :)

The blog@GLIBC_2.27 error was only due to the fact that my guest Manjaro VM’s glibc version was 2.27 and the one installed on my host Manjaro PC was 2.24.

New setup

So this morning I reinstalled entirely the arm toolchain as well as the arm glibc on my host Manjaro PC, just to make sure I had the same version on the VM and on the host PC.
With this issue in mind, I decided to also wipe the µSD card of my ARM board and put a fresh install of Debian Stretch on it.

Then I effectively installed the libs on the ARM board (and not in a fake root layout) :

apt-get install rtl-sdr librtlsdr-dev
apt-get install hackrf libhackrf-dev
mkdir tmprepo && cd tmprepo
git clone https://github.com/pothosware/SoapySDR.git
git clone https://github.com/pothosware/SoapyHackRF.git
git clone https://github.com/pothosware/SoapyRTLSDR.git
#Add sudo to `make install` and `ldconfig` commands if you are not root
cd SoapySDR/         && mkdir build && cd build && cmake .. && make && make install && ldconfig
cd ../../SoapyHackRF && mkdir build && cd build && cmake .. && make && make install && ldconfig
cd ../../SoapyRTLSDR && mkdir build && cd build && cmake .. && make && make install && ldconfig

At this point we have my host Manjaro PC containing my code as well as a folder named ARM with the following content:

ARM
├── include
│   ├── libhackrf
│   │   └── hackrf.h
│   ├── libusb-1.0
│   │   └── libusb.h
│   ├── rtl-sdr_export.h
│   ├── rtl-sdr.h
│   └── SoapySDR
│       ├── Config.h
│       ├── Config.hpp
│       ├── Constants.h
│       ├── ConverterPrimatives.hpp
│       ├── ConverterRegistry.hpp
│       ├── Device.h
│       ├── Device.hpp
│       ├── Errors.h
│       ├── Errors.hpp
│       ├── Formats.h
│       ├── Formats.hpp
│       ├── Logger.h
│       ├── Logger.hpp
│       ├── Modules.h
│       ├── Modules.hpp
│       ├── Registry.hpp
│       ├── Time.h
│       ├── Time.hpp
│       ├── Types.h
│       ├── Types.hpp
│       ├── Version.h
│       └── Version.hpp
└── lib
    ├── base
    │   ├── libdl.so.2
    │   ├── librt.so.1
    │   └── libudev.so
    ├── libSoapySDR.so
    ├── libhackrf.so
    ├── libHackRFSupport.so
    ├── librtlsdr.so
    ├── librtlsdrSupport.so
    └── libusb-1.0.so

All those files were generated on my Guest Manjaro VM. Having updated the ARM glibc on my Host Manjaro PC and with those files, I successfully compiled my software ! And It also runs correctly on the ARM board !

Afterthought

Doing some digging, I noticed some useless options in my linking command for ARM target. I used to link with -lpthread -lSoapySDR -lHackRFSupport -lhackrf -lrtlsdrSupport -lrtlsdr -lusb-1.0 -ludev -lrt -ldl but not each one is necessary. In fact, all I need is -lpthread -lSoapySDR -ludev -lrt -ldl.

As you might have guessed, now my ARM folder only needs to contain:

ARM
├── include
│   └── SoapySDR
│       ├── Config.h
│       ├── Config.hpp
│       ├── Constants.h
│       ├── ConverterPrimatives.hpp
│       ├── ConverterRegistry.hpp
│       ├── Device.h
│       ├── Device.hpp
│       ├── Errors.h
│       ├── Errors.hpp
│       ├── Formats.h
│       ├── Formats.hpp
│       ├── Logger.h
│       ├── Logger.hpp
│       ├── Modules.h
│       ├── Modules.hpp
│       ├── Registry.hpp
│       ├── Time.h
│       ├── Time.hpp
│       ├── Types.h
│       ├── Types.hpp
│       ├── Version.h
│       └── Version.hpp
└── lib
    ├── base
    │   ├── libdl.so.2
    │   ├── librt.so.1
    │   └── libudev.so
    └── libSoapySDR.so

Thank you again for your help Steve. Your suggestions helped me find my errors. Hope this thread may help someone in the same use case :)

Regards

Antony

Reply all

Reply to author

Forward

0 new messages