Undefined symbols in the _md module (v4.2.1)

[Trung Nguyen]

Hi HOOMD developers,

I have followed the instructions in the documentation for building HOOMD from source: (https://hoomd-blue.readthedocs.io/en/v4.2.1/building.html), and have had successful installation and routine runs with tag v4.2.0 (commit 4184da26c976722d9852100c14941b62a018ca05)

However, when I built and installed HOOMD with tag v4.2.1 (commit  506f9315d68a2a6ab291d9a757ee89f070b21ca9), which also finished successfully, the "import hoomd" command shows undefined symbols inside the _md module (Full trackback below):

_md.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN5hoomd2md13    PotentialPairINS0_15EvaluatorPairLJEEC2ESt10shared_ptrINS_16SystemDefinitionEES4_INS0_12NeighborListEE

When I check

nm -u _md.cpython-38-x86_64-linux-gnu.so | grep Neighbor

there are a bunch of undefined symbols from other EvaluatorPairXYZ as well.

The two builds (v4.2.0 and v4.2.1) are completely separated (different source folders and virtual envs). I have been installing using v3.8.1 and v4.2.0 without any issue. The same toolchain is used: Intel 2022.0, Intel MPI 2021.5, mkl 2020 update 1, cuda 11.5, python 3.8, anaconda 2021.05, cmake 3.19:

mkdir build && cd build

cmake ../src -DCMAKE_CXX_FLAGS=-march=native  -DCMAKE_C_FLAGS=-march=native -DENABLE_GPU=on -DENABLE_MPI=on -DBUILD_TESTING=off -DBUILD_MPCD=off

Any comments and suggestions for resolving the issue are appreciated.

Thanks,

-Trung

Traceback (most recent call last):

  2   File "run_from_restart_polymers_v4.py", line 5, in <module>
  3     import hoomd
  4   File "/project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/__init__.py", line 79, in <module>
  5     from hoomd import md
  6   File "/project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/md/__init__.py", line 22, in <module>
  7     from hoomd.md import alchemy
  8   File "/project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/md/alchemy/__init__.py", line 28, in <module>
  9     from . import methods
 10   File "/project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/md/alchemy/methods.py", line 7, in <module>
 11     from hoomd.md.alchemy.pair import AlchemicalDOF
 12   File "/project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/md/alchemy/pair.py", line 11, in <module>
 13     from hoomd.md.pair import LJGauss as BaseLJGauss
 14   File "/project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/md/pair/__init__.py", line 135, in <module>
 15     from . import aniso
 16   File "/project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/md/pair/aniso.py", line 26, in <module>
 17     from hoomd.md.pair.pair import Pair
 18   File "/project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/md/pair/pair.py", line 10, in <module>
 19     from hoomd.md import _md
 20 ImportError: /project/rcc/trung/hoomd-blue/hoomd-venv/lib/python3.8/site-packages/hoomd/md/_md.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN5hoomd2md13    PotentialPairINS0_15EvaluatorPairLJEEC2ESt10shared_ptrINS_16SystemDefinitionEES4_INS0_12NeighborListEE

[Joshua Anderson]

Trung,

Does the `nm -u` include the *precise string* from the error message?

There are no changes from 4.2.0 to 4.2.1 that would affect compilation of the _md module: https://github.com/glotzerlab/hoomd-blue/compare/v4.2.0...v4.2.1, and tests pass: https://github.com/glotzerlab/hoomd-blue/actions/runs/6381443671. Therefore, I assume that there is some difference between your two virtual environments that causes the problem. Perhaps the failing environment includes a linker that is incompatible with your compiler.

I suggest that you try 4.2.1 in the working environment:

$ cd hoomd-blue

$ git checkout v4.2.1

$ cd ..

$ cmake --build build/hoomd

$ cd build/hoomd

$ python3 -c "import hoomd"

$ python3 -m pytest hoomd

------

Joshua A. Anderson, Ph.D.

Research Area Specialist, Chemical Engineering, University of Michigan

--
You received this message because you are subscribed to the Google Groups "hoomd-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hoomd-users/CALHVP1Yw%2BGvjFYKqGGtuKhQOUrAX9%3D%3Dg6-Zmmkes6WVsZOyrLQ%40mail.gmail.com.

[Trung Nguyen]

Hi Josh,

thanks for the suggestions. 

1) Yes, the `nm -u` include the *precise string* from the error message.

2) I rebuilt 4.2.1 in the working environment as instructed. There are no such undefined symbols errors anymore. I can confirm that version is 4.2.1 (python3 -c "import hoomd; print(hoomd.version.version)".

At this point, the issue can be marked as resolved, but the reason is still unknown. Something in the failing env that prevented the linker from putting the relevant objects to the _md.so target.

Thanks again,

-Trung

To view this discussion on the web visit https://groups.google.com/d/msgid/hoomd-users/60BF7A35-29CB-421D-A4FD-24B1BF9B2DC0%40umich.edu.

[Joshua Anderson]

Trung,

Good that you got it working. I'm not surprised. You mentioned you are using an anacnoda environment. I have seen *many* issues like this over the years when anaconda provided compilers and/or linkers are installed. To find the problem package, check the list of packages you have installed in the non-working environment and compare to the working one. Uninstall any that provide compiler or linker tools one by one until the error goes away.

You will need to rerun cmake in an empty build directory each time, as cmake discovers what compilers and linkers to use on first configure.

------

Joshua A. Anderson, Ph.D.

Research Area Specialist, Chemical Engineering, University of Michigan

To view this discussion on the web visit https://groups.google.com/d/msgid/hoomd-users/CALHVP1YRn3xXazPsu1%2BGpiRDST6Adv8dwinByDWYYehrqqH6RA%40mail.gmail.com.