Import error

Konstantinos Karapiperis

unread,

Dec 9, 2020, 6:24:42 PM12/9/20

to freud-users

Hi all,

I am working on Ubuntu 18.04 and installing freud and its dependencies via pip.

After installing and trying to import freud in Python 3.6, I get:

libfreud.so: undefined symbol: _ZN3tbb19task_scheduler_init9terminateEv

Any suggestions on how to resolve this?

Thanks,

Konstantinos

Bradley Dice

unread,

Dec 16, 2020, 3:36:16 PM12/16/20

to freud-users

Hi Konstantinos,

I tried to replicate this setup in a Docker container. I used Ubuntu 18.04 with Python 3.6. Do you have the TBB library installed? Try "sudo apt install libtbb2" or "conda install tbb" or "pip install tbb" to get the TBB library. TBB must be installed but we don't include TBB as a pip requirement because it can cause problems if the user wants to use a system-installed TBB library (using multiple TBB libraries at the same time may cause conflicts with thread pools and resource allocation).

I assume you're using the wheel, and not building from source. If you are building from source (which you could see if pip fetches the .tar.gz), then you will also need TBB headers installed. You can run "sudo apt install libtbb-dev" to get the TBB headers, or "conda install tbb-devel". I don't think the headers are available from pip / PyPI.

If you're still having issues, could you please list the full versions of (and how you installed) the dependencies listed in our installation instructions, and share the full output of the installation command? https://freud.readthedocs.io/en/stable/gettingstarted/installation.html

Thanks,

Bradley

Konstantinos Karapiperis

unread,

Dec 19, 2020, 4:33:01 PM12/19/20

to freud-users

Hi Bradley,

thanks for your answer. I have both the TBB library and the TBB headers installed.

I am using the wheel. Still having the same issues. Here is the output of my installation:

konstantinos@XPS:~$ pip3 install freud-analysis

Collecting freud-analysis

Collecting rowan>=1.2 (from freud-analysis)

Using cached https://files.pythonhosted.org/packages/1e/c9/1db928d23c74318af701d94cc60f880ebcff40c1493e0bf05d5a05cb74a4/rowan-1.3.0.post1-py2.py3-none-any.whl

Collecting cython>=0.29 (from freud-analysis)

Using cached https://files.pythonhosted.org/packages/19/49/91ebe4a00bf894d08dd9680cd9dfb05936eb2848eebd9402b43885aa74cf/Cython-0.29.21-cp36-cp36m-manylinux1_x86_64.whl

Collecting scipy>=1.1 (from freud-analysis)

Using cached https://files.pythonhosted.org/packages/c8/89/63171228d5ced148f5ced50305c89e8576ffc695a90b58fe5bb602b910c2/scipy-1.5.4-cp36-cp36m-manylinux1_x86_64.whl

Collecting numpy>=1.14 (from freud-analysis)

Using cached https://files.pythonhosted.org/packages/a6/fc/36e52d0ae2aa502b211f1bcd2fdeec72d343d58224eabcdddc1bcb052db1/numpy-1.19.4-cp36-cp36m-manylinux1_x86_64.whl

Installing collected packages: numpy, rowan, cython, scipy, freud-analysis

Successfully installed cython-0.29.21 freud-analysis-2.4.1 numpy-1.19.4 rowan-1.3.0.post1 scipy-1.5.4

and of the problem:

konstantinos@XPS:~$ python3.6

Python 3.6.9 (default, Oct 8 2020, 12:12:24)

[GCC 8.4.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import freud

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

File "/home/konstantinos/.local/lib/python3.6/site-packages/freud/__init__.py", line 5, in <module>

from . import cluster

ImportError: /home/konstantinos/.local/lib/python3.6/site-packages/freud/libfreud.so: undefined symbol: _ZN3tbb19task_scheduler_init9terminateEv

If instead of pip, I use conda, then I cannot install freud. Conda just gets stuck here:

(base) konstantinos@XPS:~$ conda install -c conda-forge freud

Solving environment\

Thanks again for the help,

Konstantinos

Bradley Dice

unread,

Dec 20, 2020, 9:45:28 PM12/20/20

to freud-users

Konstantinos,

I was not able to replicate this failure in a Docker container for Ubuntu 18.04 and Python 3.6 when installing TBB with apt and installing freud 2.4.1 with pip. I have some ideas below for how you might be able to solve this, but I would need more information to solve the specific failure you're seeing. I am interested in diagnosing your problem so that we can solve the installation issue for future users.

Diagnosing the failure you're seeing:

1. Can you clarify how you installed the TBB library? What version of TBB do you have installed?

2. Paste the output of: "ldd /home/konstantinos/.local/lib/python3.6/site-packages/freud/libfreud.so". This should print out the library paths that are dependencies of libfreud.so. Check if the TBB path that it finds actually exists on your system.

3. We recently changed the build system in freud v2.4.0. Can you try to install with "pip install freud-analysis==2.3.0" and see if that works? That would help determine if it's caused by the change in our build system. (Please run the ldd command from step 2 on the latest version, 2.4.1 -- not 2.3.0.)

Possible workarounds if we can't solve your TBB linking problem:

You could try cloning the repository and installing from source following these instructions, or install from pip with the "--no-binary" flag which would use the source distribution (instead of a wheel distribution) which should find and build with your system's TBB library (requires TBB headers): "pip install freud-analysis --no-binary"

The installation with conda would probably be fixed by one of the following:

- Try installing into a new conda environment instead of "base"

- Upgrade your conda installation if it's running an older version of conda (bugs have been fixed and performance has improved)

- Try installing using mamba, which is much faster than conda at solving environments

You can also use a Docker or Singularity software container that includes freud: https://glotzerlab-software.readthedocs.io/en/stable/

I hope we can identify the root cause and find a good solution. Thanks for your help.

Best,

Bradley

Konstantinos Karapiperis

unread,

Dec 22, 2020, 7:07:08 PM12/22/20

to freud-users

Thanks for your help, Bradley.

1. I installed tbb using: apt-get install libtbb2 libtbb2-dev. Here is the output

Reading package lists... Done

Building dependency tree

Reading state information... Done

libtbb2 is already the newest version (2017~U7-8).

libtbb-dev is already the newest version (2017~U7-8).

The following packages were automatically installed and are no longer required:

apt-clone archdetect-deb btrfs-tools chromium-codecs-ffmpeg-extra dmraid

gir1.2-geocodeglib-1.0 gir1.2-timezonemap-1.0 gir1.2-xkl-1.0 kpartx

kpartx-boot libdebian-installer4 libdmraid1.0.0.rc16 libfwup1 libido3-0.1-0

libllvm8 libllvm9 libtimezonemap-data libtimezonemap1

linux-oem-osp1-headers-5.0.0-1031 linux-oem-osp1-headers-5.0.0-1052

linux-oem-osp1-headers-5.0.0-1059 linux-oem-osp1-headers-5.0.0-1063

linux-oem-osp1-headers-5.0.0-1065 linux-oem-osp1-headers-5.0.0-1067

linux-oem-osp1-headers-5.0.0-1068 linux-oem-osp1-headers-5.0.0-1069

python3-icu python3-pam rdate ubuntu-web-launchers

2. Here is the output of ldd:

linux-vdso.so.1 (0x00007fff205be000)

libtbb.so.12 => /usr/local/lib/libtbb.so.12 (0x00007f228dc7c000)

libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f228d8f3000)

libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f228d555000)

libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f228d33d000)

libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f228cf4c000)

libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f228cd48000)

libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f228cb29000)

/lib64/ld-linux-x86-64.so.2 (0x00007f228e6ad000)

Indeed libtbb.so.12 is in the directory shown above.

3. Installing freud v2.3.0 as recommended, actually solves the problem!

Thanks again,

Konstantinos

vramasub

unread,

Jan 2, 2021, 3:47:43 PM1/2/21

to freud-users

Hi Konstantinos,

Sorry for not getting a chance to look at this sooner, I was in the middle of a move when you first posted this. From that final ldd output, it looks like your freud is finding the TBB you installed via apt-get. If you're installing freud from PyPI (pip install freud-analysis) TBB actually does not need to be installed separately; the wheels on PyPI embed any dynamic libraries directly into the wheel precisely to avoid this requirement. Having locally installed TBB versions should not cause any problems, but based on your ldd output for some reason freud is finding your installation of TBB instead of the version in the wheel, and since we build TBB from source as part of our deployment the TBB version that we have will be newer than the TBB you install via apt-get, so you run into ABI incompatibilities when the older library is loaded.

As Bradley mentioned, we recently rewrote our build system. I just downloaded the Linux wheels to double check, and the TBB libraries are correctly being embedded, but libfreud sees system TBB rather than the TBB in the wheel. I don't observe this problem on Mac, so I would guess that it's either an issue with the way that our new CMake configuration is setting the RPath on Linux, or a problem with the way in which we modify the RPath for the wheel. I'll try and identify the cause of the problem, but for now I'm glad 2.3 is working for you!

Best,

Vyas

alienhunter_D

unread,

Feb 2, 2021, 9:16:16 AM2/2/21

to freud-users

Hello I have the same issue with a fresh install ubuntu 20.04 Using version 2.3 solves the problem.

Greetings Martin

Bradley Dice

unread,

Feb 2, 2021, 2:17:29 PM2/2/21

to freud-users

Thanks for reporting -- I may have identified the problem, which I documented in this issue: https://github.com/glotzerlab/freud/issues/720

However, I am not sure how to fix it yet. We'll need to carefully investigate our build scripts and the deployment process that generates the wheels uploaded to PyPI.

Here are a few workarounds for now, which I will document in that issue:

1. Install from the source distribution on PyPI. This requires a system installation of TBB to build the package from source.

sudo apt install libtbb2 libtbb-dev

pip3 install freud-analysis --no-binary freud-analysis

2. Install from conda-forge.

conda install -c conda-forge freud

3. Install an older binary that uses the correct RPATHs.

pip3 install freud-analysis==2.3.0

Hope this helps. We'll try to fix this in the next release.

Best,

Bradley

Bradley Dice

unread,

Mar 24, 2021, 3:30:20 PM3/24/21

to freud-users

For users that were facing issues with Linux builds and TBB, those problems should now be fixed. I helped with a pull request to auditwheel, which was incorrectly patching freud's wheels and not including the correct linkages to libfreud and TBB. This patch has been merged upstream and will eventually be included in the manylinux images used by many package developers for making PyPI releases. Thank you to Konstantinos, Martin, Vyas, and Joshua Anderson for help in identifying the problem!