May I have your help about an issue during installing HOOMD?

[Wusheng Zhang]

Dear HOOMD users,

I am installing HOOMD GPU MPI version on the HPC cluster of my university. After starting an interactive session with requesting one GPU card, and loading the module cuda/8.0. gcc/6.3.0, cmake/3.5.2, and python/2.7.12, I compiled openmpi 2.1.1 with cuda locally, and complied fftw 3.3.6 enable mpi locally. Then I add the the following lines to ~/.bashrc and ~/.bash_profile, and do ". ~/.bashrc" to update the new path and dynamic library path.

export PATH=$HOME/software/openmpi_gpu/bin:$PATH

export LD_LIBRARY_PATH=$HOME/software/openmpi_gpu/lib:$LD_LIBRARY_PATH

export LD_LIBRARY_PATH=$HOME/software/fftw/lib:$LD_LIBRARY_PATH

Then I used the command below to compile hoomd:

git clone https://bitbucket.org/glotzer/hoomd-blue.git

cd hoomd-blue

mkdir build

cd build

cmake ../ -DCMAKE_INSTALL_PREFIX=$HOME/programs/hoomd-blue/build

make all

make install

I met three problems during the procedures above:

1. During "cmake ../ -DCMAKE_INSTALL_PREFIX=$HOME/programs/hoomd-blue/build": it returns that cannot find fftw, but I have fftw 3.3.6 compiled. Do I need fftw 2.2.5 instead?

2. During "make all": it always says

"/nas/longleaf/home/minzhi/hoomd-blue/hoomd/extern/kernels/segreducecsr.cuh:394:8: warning: ‘auto_ptr’ is deprecated (declared at /usr/include/c++/4.8.2/backward/auto_ptr.h:87) [-Wdeprecated-declarations]

  CudaContext& context)"

But I am using gcc/6.3.0 which is >4.8.5

3. During "make install": I got the following stuff after [100%].

[100%] Building NVCC (Device) object hoomd/example_plugin/CMakeFiles/cuda_compile.dir/cuda_compile_generated_ExampleUpdater.cu.o

[100%] Linking CXX shared library _example_plugin.so

[100%] Built target _example_plugin

Install the project...

-- Install configuration: "Release"

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/HOOMDCFlagsSetup.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/HOOMDCommonLibsSetup.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/HOOMDCUDASetup.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/test.cc

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/HOOMDMacros.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/HOOMDMPISetup.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/HOOMDNumpySetup.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/HOOMDOSSpecificSetup.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/HOOMDPythonSetup.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/FindGit.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/GetGitRevisionDescription.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/Findlibgetar.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/FindThrust.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/CMake_build_options.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/CMake_install_options.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/CMake_preprocessor_flags.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/CMake_src_setup.cmake

-- Installing: /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/CMake_version_options.cmake

CMake Error at hoomd/cmake_install.cmake:46 (file):

  file INSTALL cannot find

  "/nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/_hoomd.so".

Call Stack (most recent call first):

  cmake_install.cmake:65 (include)

make: *** [install] Error 1

May I have your suggestions about this issue?

Besides, if I install an anaconda2 on the HPC cluster, and then I load one GPU card, and with all the modules described loaded above, and have the openmpi_gpu and fftw 3.3.6 compiled locally, will I have an ENABLE_MPI_CUDA version of HOOMD by conda install command as below? Or if I want ENABLE_MPI_CUDA version of HOOMD, I must compile by myself?

$ conda config --add channels glotzer

$ conda install hoomd

Thank you very much.

Best regards,

Wusheng

[Jens Glaser]

Wusheng,

I am installing HOOMD GPU MPI version on the HPC cluster of my university. After starting an interactive session with requesting one GPU card, and loading the module cuda/8.0. gcc/6.3.0, cmake/3.5.2, and python/2.7.12, I compiled openmpi 2.1.1 with cuda locally, and complied fftw 3.3.6 enable mpi locally. Then I add the the following lines to ~/.bashrc and ~/.bash_profile, and do ". ~/.bashrc" to update the new path and dynamic library path.

export PATH=$HOME/software/openmpi_gpu/bin:$PATH

export LD_LIBRARY_PATH=$HOME/software/openmpi_gpu/lib:$LD_LIBRARY_PATH

export LD_LIBRARY_PATH=$HOME/software/fftw/lib:$LD_LIBRARY_PATH

Then I used the command below to compile hoomd:

git clone https://bitbucket.org/glotzer/hoomd-blue.git

cd hoomd-blue

mkdir build

cd build

cmake ../ -DCMAKE_INSTALL_PREFIX=$HOME/programs/hoomd-blue/build

make all

make install

I met three problems during the procedures above:

1. During "cmake ../ -DCMAKE_INSTALL_PREFIX=$HOME/programs/hoomd-blue/build": it returns that cannot find fftw, but I have fftw 3.3.6 compiled. Do I need fftw 2.2.5 instead?

Are you referring to MKL? What is the exact error message?

fftw is not used by HOOMD at all. MKL is optional and not really needed (except if you are doing multi-CPU PPPM). And I haven’t tested that in a long time.

By the way, do set -D ENABLE_MPI=ON and -DENABLE_CUDA=ON for this version. The defaults have changed.

2. During "make all": it always says

"/nas/longleaf/home/minzhi/hoomd-blue/hoomd/extern/kernels/segreducecsr.cuh:394:8: warning: ‘auto_ptr’ is deprecated (declared at /usr/include/c++/4.8.2/backward/auto_ptr.h:87) [-Wdeprecated-declarations]

  CudaContext& context)"

But I am using gcc/6.3.0 which is >4.8.5

Well, that’s just a warning. Safely ignore it.

Hmm, that’s strange. Why would it not find the library after correct compilation? Is the cmake version incompatible

perhaps? Just guessing, Joshua may chime in. Does the compiler output indicate any other *errors* or does it indicate

somewhere in the long output that _hoomd.so has been built? In any case, it shouldn’t get to this point without that the library

has been built.

May I have your suggestions about this issue?

Besides, if I install an anaconda2 on the HPC cluster, and then I load one GPU card, and with all the modules described loaded above, and have the openmpi_gpu and fftw 3.3.6 compiled locally, will I have an ENABLE_MPI_CUDA version of HOOMD by conda install command as below? Or if I want ENABLE_MPI_CUDA version of HOOMD, I must compile by myself?

Without much conda knowledge, you don’t need ENABLE_MPI_CUDA, only ENABLE_CUDA and ENABLE_MPI.

ENABLE_MPI_CUDA is for using a GPU aware MPI port, which is only faster in some corner cases.

$ conda config --add channels glotzer

$ conda install hoomd

Thank you very much.

Best regards,

Wusheng

- Jens

-- 
You received this message because you are subscribed to the Google Groups "hoomd-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users...@googlegroups.com.
To post to this group, send email to hoomd...@googlegroups.com.
Visit this group at https://groups.google.com/group/hoomd-users.
For more options, visit https://groups.google.com/d/optout.

[Joshua Anderson]

Does the file /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/_hoomd.so exist? (check with ls -l /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/)

If it doesn't, that would be very strange. Your output indicates a successful build and link of _example_plugin.so which links to _hoomd.so.

I don't recommend using conda on clusters. Build HOOMD with the native cluster MPI library to take advantage of the high performance communication fabric.

------
Joshua A. Anderson, Ph.D.
Research Area Specialist, Chemical Engineering, University of Michigan
http://www-personal.umich.edu/~joaander/

Wusheng,

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.
To post to this group, send email to hoomd-users@googlegroups.com.

Visit this group at https://groups.google.com/group/hoomd-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "hoomd-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.

[Wusheng Zhang]

Hi Jens,

Thank you very much for your patient help.

1. Yes, MKL not found. In fact, I even do not found MKL pre-installed in our clusters. But since you mentioned that I do not need it. I will ignore it.

2. I see. I am just curious about that warning because I am using something newer than gcc/4.8.5 but still get that warning.

3. There is no other error about "_hoomd.so" in the output before. But what I intend to do is to install hoomd 2.2.0 and see what happened.

4. Here is the new question: when you say " do set -D ENABLE_MPI=ON and -DENABLE_CUDA=ON for this version", do you mean that to Hoomd 2.1.9, I should use the following command right? I just want to make sure because I am not so familiar with cmake flags. And besides, if I need GPU MPI version of hoomd 2.2.0, do I need to apply the same command as below?

cmake ../-D ENABLE_MPI=ON and -DENABLE_CUDA=ON -DCMAKE_INSTALL_PREFIX=$HOME/programs/hoomd-blue/build

Thank you very much.

Best regards,

Wusheng

在 2017年9月7日星期四 UTC-4下午10:45:33，Jens写道：

[Wusheng Zhang]

Hi Joshua,

I checked, and /nas/longleaf/home/minzhi/hoomd-blue/build/hoomd/_hoomd.so does not exist. I am trying to compile the new version 2.2.0 and to see what will happen. Thank you very much for your help.

Best regards,

Wusheng

在 2017年9月8日星期五 UTC-4上午7:16:27，Joshua A. Anderson写道：

Wusheng,

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users...@googlegroups.com.

To post to this group, send email to hoomd...@googlegroups.com.
Visit this group at https://groups.google.com/group/hoomd-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "hoomd-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users...@googlegroups.com.

[Michael Howard]

Did you try installing somewhere other than your build directory (i.e., different CMAKE_INSTALL_PREFIX)? It looks like the failure is on the install step rather than the make, so maybe CMake does something strange if you are trying to install into where the build objects already are.

And, auto_ptr was deprecated in newer versions of gcc, and you're using a very recent version, which is why Jena said you can safely ignore this.

Regard,
Mike

[Wusheng Zhang]

Hi Mike,

Thank you very much! I tried different CMAKE_INSTALL_PREFIX location, but a new bug happened as shown in the reply below.

Best regards,

Wusheng

在 2017年9月8日星期五 UTC-4下午9:58:18，Michael Howard写道：

[Wusheng Zhang]

Hi Jens,

I tried to install the latest version of hoomd, and there no any error or warning during the process as below

cmake ../ -DENABLE_MPI=ON -DENABLE_CUDA=ON -DCMAKE_INSTALL_PREFIX=/nas/longleaf/home/minzhi/anaconda2/lib/python2.7/site-packages

make -j4

make install

except the error below. The error below happened during "make install", and after 100% finished, it tried to create the directory in the /usr/local/ which need root authority. However, such problem will never happen when I did not load GPU card.

[ 94%] Building NVCC (Device) object hoomd/metal/CMakeFiles/cuda_compile.dir/cuda_compile_generated_EAMForceGPU.cu.o

[ 94%] Linking CXX shared library _metal.so

[ 95%] Built target _metal

[ 95%] Building NVCC (Device) object hoomd/dem/CMakeFiles/cuda_compile.dir/templateInstances/cuda_compile_generated_DEM3DForce_NF_WCA.cu.o

[ 95%] Building NVCC (Device) object hoomd/dem/CMakeFiles/cuda_compile.dir/templateInstances/cuda_compile_generated_DEM2DForce_NF_SWCA.cu.o

[ 96%] Building NVCC (Device) object hoomd/dem/CMakeFiles/cuda_compile.dir/templateInstances/cuda_compile_generated_DEM2DForce_NF_WCA.cu.o

[ 96%] Building NVCC (Device) object hoomd/dem/CMakeFiles/cuda_compile.dir/templateInstances/cuda_compile_generated_DEM3DForce_NF_SWCA.cu.o

[ 96%] Linking CXX shared library _dem.so

[ 97%] Built target _dem

[ 97%] Built target copy_dem

[ 97%] Building NVCC (Device) object hoomd/example_plugin/CMakeFiles/cuda_compile.dir/cuda_compile_generated_ExampleUpdater.cu.o

[ 98%] Linking CXX shared library _example_plugin.so

[ 98%] Built target _example_plugin

[100%] Built target copy_example_plugin

Install the project...

-- Install configuration: "Release"

CMake Error at cmake_install.cmake:36 (file):

  file cannot create directory: /usr/local/hoomd.  Maybe need administrative

  privileges.

make: *** [install] Error 1

This is really wired. Once I do not load the GPU card (just start an interactive session with CPUs), the compilation will be successful. But once I load the GPU card, the compilation will result as above: it tried to write to the /usr/local/hoomd which I need root authority, but just as I mentioned, I assign the CMAKE_INSTALL_PREFIX to my own home directory. So may I have your suggestion about this? Thank you very much.

Best regards,

Wusheng

在 2017年9月7日星期四 UTC-4下午10:45:33，Jens写道：

Wusheng,

[Jens Glaser]

Wusheng,

you will need root priority, as you say, to install into /usr/local.

I suspect your order of arguments to cmake is wrong.

Try putting the ../ at the end of the command line, not the beginning.

Jens

[Joshua Anderson]

Inspect CMakeCache.txt and/or view the cache with the ccmake ncurses gui program to verify that the install prefix is set as intended.

------
Joshua A. Anderson, Ph.D.
Research Area Specialist, Chemical Engineering, University of Michigan
http://www-personal.umich.edu/~joaander/

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.
To post to this group, send email to hoomd-users@googlegroups.com.

Visit this group at https://groups.google.com/group/hoomd-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "hoomd-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.

[Wusheng Zhang]

Hi Jens and Joshua,

I finally got hoomd blue installed on the HPC cluster successfully. The problems now I found are:

1. Here is a hidden directory ${HOME}/.local/, and here is a broken python 2.7 in this directory; this should be the main problem;

2. I compiled the openmpi with cuda8.0 and intel 17.2, but the C++ compiler the hoomd will use is gcc. But I think this might not be a problem. Anyway, I recompile the openmpi with cuda and gcc 4.8.5.

I first install an anaconda2 in my home directory and set it as my default python. For users' reference, after start an interactive session with requesting one CPU and one GPU card, my installation process is,

module load cuda/8.0

module load cmake/3.5.2

module load openmpi_gcc/local_4.8.5_cuda8.0 #this is the openmpi compiled with cuda 8.0 and gcc 4.8.5

conda install sphinx pkg-config sqlite

git clone https://bitbucket.org/glotzer/hoomd-blue

cd hoomd-blue

mkdir build

cd build

cmake ../ -DCMAKE_INSTALL_PREFIX=/nas/longleaf/home/minzhi/anaconda2/lib/python2.7/site-packages -DENABLE_MPI=ON -DENABLE_CUDA=ON -DENABLE_MPI_CUDA=ON

make -j1

make install

Then save set the python path in the .bashrc, and activate it.

export PYTHONPATH=$PYTHONPATH:/nas/longleaf/home/minzhi/anaconda2/lib/python2.7/site-packages

Then I tried to submit a job to run the example lj.py shown in your website on 2 GPU cards, and two CPUs. What was printed on the screen is

HOOMD-blue v2.2.0-2-g75f97d0 CUDA (8.0) DOUBLE HPMC_MIXED MPI MPI_CUDA SSE SSE2 

Compiled: 09/15/2017

Copyright 2009-2017 The Regents of the University of Michigan.

-----

You are using HOOMD-blue. Please cite the following:

* J A Anderson, C D Lorenz, and A Travesset. "General purpose molecular dynamics

  simulations fully implemented on graphics processing units", Journal of

  Computational Physics 227 (2008) 5342--5359

* J Glaser, T D Nguyen, J A Anderson, P Liu, F Spiga, J A Millan, D C Morse, and

  S C Glotzer. "Strong scaling of general-purpose molecular dynamics simulations

  on GPUs", Computer Physics Communications 192 (2015) 97--107

-----

notice(2): This system is not compute exclusive, using local rank to select GPUs

HOOMD-blue is running on the following GPU(s):

Rank 0:  [0]      GeForce GTX 1080  20 SM_6.1 @ 1.73 GHz, 8113 MiB DRAM

Rank 1:  [1]      GeForce GTX 1080  20 SM_6.1 @ 1.73 GHz, 8113 MiB DRAM

lj.py:006  |  hoomd.init.create_lattice(unitcell=hoomd.lattice.sc(a=2.0, type_name='A'), n=10)

lj.py:006  |  hoomd.init.create_lattice(unitcell=hoomd.lattice.sc(a=2.0, type_name='A'), n=10)

HOOMD-blue is using domain decomposition: n_x = 1 n_y = 1 n_z = 2.

1 x 1 x 2 local grid on 1 nodes

notice(2): Group "all" created containing 1000 particles

lj.py:008  |  nl = md.nlist.cell()

lj.py:009  |  lj = md.pair.lj(r_cut=3.0, nlist=nl)

lj.py:010  |  lj.pair_coeff.set('A', 'A', epsilon=1.0, sigma=1.0)

lj.py:012  |  all = hoomd.group.all();

lj.py:013  |  md.integrate.mode_standard(dt=0.005)

lj.py:014  |  hoomd.md.integrate.langevin(group=all, kT=1.2, seed=4)

notice(2): integrate.langevin/bd is using specified gamma values

lj.py:016  |  hoomd.run(10e3)

notice(2): -- Neighborlist exclusion statistics -- :

notice(2): Particles with 0 exclusions             : 1000

notice(2): Neighbors included by diameter          : no

notice(2): Neighbors excluded when in the same body: no

** starting run **

Time 00:00:02 | Step 10000 / 10000 | TPS 4667.82 | ETA 00:00:00

Average TPS: 4667.69

---------

-- Neighborlist stats:

1046 normal updates / 34 forced updates / 0 dangerous updates

n_neigh_min: 7 / n_neigh_max: 40 / n_neigh_avg: 20.9752

shortest rebuild period: 7

-- Cell list stats:

Dimension: 5, 5, 3

n_min    : 2 / n_max: 20 / n_avg: 11.0267

** run complete **

Thanks again.

Best regards,

Wusheng

在 2017年9月13日星期三 UTC-4上午7:04:51，Joshua A. Anderson写道：

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users...@googlegroups.com.