Running FDS with OpenMPI in Linux system

1,042 views
Skip to first unread message

Daniel

unread,
Oct 6, 2016, 11:06:28 AM10/6/16
to FDS and Smokeview Discussions
Dear all,
Recently I installed FDS 6.5.2 in CentOS system. Before it, I have installed OpenMPI 1.8.4 and tested "mpirun --version" command. It worked well.
Then I followed the procedure from the below website and compiled the FDS installation script.
But after I typed "fds" command, it showed me "fds: /usr/lib64/libnuma.so.1: no version information available (required by fds)". Where does this problem come from?

Then I run some serial cases, which worked well. But when I run the parallel case, it showed me some errors.
I have put the error in the attachment. The 1st picture is the error. The 2nd one is the ".bashrc" file. And 3rd one is the ".bashrc_fds" file.
I am new to the parallel computation in linux system. So if anyone has some ideas, please give me a hand. 
Thank you so much!
Error.PNG
bashrc.PNG
bashrc_fds.PNG

Kevin

unread,
Oct 6, 2016, 12:43:46 PM10/6/16
to FDS and Smokeview Discussions
I've never seen that error message before. How did you run the parallel job?

Daniel

unread,
Oct 6, 2016, 12:54:11 PM10/6/16
to FDS and Smokeview Discussions
Hi, Kevin. Thanks for your response.
I used the command "mpirun -np 3 fds tunnel.fds".
I also have another question. In my ".bashrc" file, there are no values for "MPIDIST_ETH" and "MPIDIST_IB". 
Do you know why? 

在 2016年10月7日星期五 UTC+9上午1:43:46，Kevin写道：

Kevin

unread,
Oct 6, 2016, 12:59:07 PM10/6/16
to FDS and Smokeview Discussions
#FDS environment -----------------------
export MPIDIST_ETH=/shared/openmpi_64
export MPIDIST_IB=/shared/openmpi_64ib
source ~/.bashrc_fds $MPIDIST_IB
#FDS -----------------------------------

This is what I have. ETH stands for ethernet and IB for Infiniband. I am using Infiniband.

Daniel

unread,
Oct 7, 2016, 10:00:58 AM10/7/16
to FDS and Smokeview Discussions
Hello, Kevin.
I opened the "FDS_6.5.2-SMV_6.3.12_linux64.sh" file and found below commands:
mpipath=
mpipatheth=
mpipathib=
if [ -d /shared/openmpi_64 ] ; then
    mpipatheth=/shared/openmpi_64
    mpipath=$MPIDIST_ETH
    mpipath2=\$MPIDIST_ETH
if
if [ -d /shared/openmpi_64ib ] ; then
    mpipathib=/shared/openmpi_64ib
    mpipath=$MPIDIST_IB
    mpipath2=\$MPIDIST_IB
if
The default directories for MPIDIST_ETH and MPIDIST_IB are both in the "shared" directory created in root directory.
I don't own the root password. So I installed my OpenMPI _1.8.4 in my own home directory (without "sudo" command during my installation).
Here I want to know whether it will work well or not after I change the default directories for MPIDIST_ETH and MPIDIST_IB to the directory where I have installed my OpenMPI.
Please give me some advice. Many thanks!



在 2016年10月7日星期五 UTC+9上午1:59:07，Kevin写道：

Glenn Forney

unread,
Oct 7, 2016, 10:22:49 AM10/7/16
to fds...@googlegroups.com
I would suggest that you not change the installation script.  Run it as is .  Then edit your .bashrc file and change the lines

export MPIDIST_ETH=/shared/openmpi_64
export MPIDIST_IB=/shared/openmpi_64ib
source ~/.bashrc_fds $MPIDIST_IB

so that MPIDIST_ETH points to where you have installed your openmpi library.  I assume you do not have infiniband networking hardware.  I have done something similar on systems where I do not have root privilidges.



--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/2ef8b5ef-105b-4f4b-85a1-dd4c8c485667%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Glenn Forney

o...@aquacoustics.biz

unread,
Oct 7, 2016, 6:22:19 PM10/7/16
to FDS and Smokeview Discussions
Dear Daniel,

going back to your original question, libnuma is a required package for Mellanox OFED.  I suspect that, subject to your flavor of Infiniband, the package may either be missing, the wrong version or has become corrupted.  Please refer to your OFED installation and release notes for required OFED support packages.  libnuma1 is available for download from either the Software Center (Ubuntu) or your favorite Linux package manager (synaptic).

t.
 
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/2ef8b5ef-105b-4f4b-85a1-dd4c8c485667%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Glenn Forney

Daniel

unread,
Oct 9, 2016, 7:30:31 AM10/9/16
to FDS and Smokeview Discussions
Dear Glenn Forney,
Today I tried to install the FDS again and edit the .bashrc file.
I use ethenet networking hardware. So I changed the MPIDIST_ETH to the directory where I have installed my openmpi library.
But when I run the parallel job, it still showed me the same error I have posted initially.(the "Error" picture)

在 2016年10月7日星期五 UTC+9下午11:22:49，Glenn Forney写道：
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/2ef8b5ef-105b-4f4b-85a1-dd4c8c485667%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Glenn Forney

Glenn Forney

unread,
Oct 9, 2016, 9:25:14 AM10/9/16
to fds...@googlegroups.com
show the lines where you source .bashrc_fds and the lines just before it

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/c6b840cf-fe06-40e6-96df-bd1e73c25051%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Glenn Forney

Daniel

unread,
Oct 9, 2016, 10:21:44 AM10/9/16
to FDS and Smokeview Discussions
I have added the "source ~/.bashrc_fds" command in the .bashrc file. I put it in the attachment.
I am guessing there may be something wrong with my openmpi. Because I noticed that the installation methods of openmpi for infiniband and ethernet networking are different.
As I have known, our supercomputer uses the ethernet. I used the command below to install openmpi:
./configure --prefix /shared/openmpi_64 CC=icc CXX=icpc F77=ifort FC=ifort CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64
make
make install

Is anything wrong with this installation method?

在 2016年10月9日星期日 UTC+9下午10:25:14，Glenn Forney写道：



--
Glenn Forney
bashrc.PNG

Glenn Forney

unread,
Oct 9, 2016, 10:29:59 AM10/9/16
to fds...@googlegroups.com
don't see anything wrong with that.  Look at fds/Build/Scripts/OPENMPI/184INTEL_ETH in your fds repo to see how we built our ethernet openmpi library.  I asume you define the intel fortran environment in your .bashrc file?

in your .bashrc file, show the lines where you source .bashrc_fds and the lines just before it

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/94642006-2020-4db8-ba20-f7af53ef0560%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Glenn Forney

Daniel

unread,
Oct 9, 2016, 10:53:34 AM10/9/16
to FDS and Smokeview Discussions
I'm sorry, but I am not sure about the intel fortran environment.
Before the installation of openmpi, I just loaded the "intel/intel-11.1" module file.
I have read 184INTEL_ETH in fds repo, I noticed that in "CONFIGURE.sh" file the default directory in "./configure" line is "/shared/openmpi_64".
But my installation of openmpi is in another directory.
Is there anything wrong with this?

在 2016年10月9日星期日 UTC+9下午11:29:59，Glenn Forney写道：



--
Glenn Forney

Glenn Forney

unread,
Oct 9, 2016, 11:10:24 AM10/9/16
to fds...@googlegroups.com
type ifort and icc at the command line does it work? If so, then your intel fortran environment has been set up

in your .bashrc file, report back the following:  the line where you source .bashrc_fds and the lines just before it 



To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/cb596cbd-a561-4899-91a7-04bc26aff815%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Glenn Forney

Daniel

unread,
Oct 9, 2016, 11:34:25 AM10/9/16
to FDS and Smokeview Discussions
Yes, it really works when I type ifort and icc.
In my .bashrc file, below are the lines where I source .bashrc_fds:
export MPIDIST_ETH=/pwork01/p679fen/applications/openmpi/1.8.4
export MPIDIST_IB=
source ~/.bashrc_fds $MPIDIST_ETH

在 2016年10月10日星期一 UTC+9上午12:10:24，Glenn Forney写道：



--
Glenn Forney

Glenn Forney

unread,
Oct 9, 2016, 8:48:36 PM10/9/16
to fds...@googlegroups.com
type the following, what is reported?
which mpirun

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/fee41415-8723-489b-a0be-c7a6a6b3bffe%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Glenn Forney

Daniel

unread,
Oct 10, 2016, 1:27:09 AM10/10/16
to FDS and Smokeview Discussions
It showed "/pwork01/p679fen/applications/openmpi/1.8.4/bin/mpirun".
That is the directory where I installed my openmpi.

在 2016年10月10日星期一 UTC+9上午9:48:36，Glenn Forney写道：



--
Glenn Forney

Daniel

unread,
Oct 10, 2016, 9:59:48 PM10/10/16
to FDS and Smokeview Discussions
Thanks for all your help, now I have solved above problems.
But I met a new problem. In our supercomputer, we use LoadLeveler to submit and manage our job.
I wrote the script and submitted the parallel fds job. But it showed me an error:
"error while loading shared libraries: libtorque.so.2: cannot open shared object file: No such file or directory".
By the way, I have added the "FDS6/bin/LIB64" directory to the LD_LIBRARY_PATH.
Could anyone help me with this?

在 2016年10月10日星期一 UTC+9下午2:27:09，Daniel写道：

Glenn Forney

unread,
Oct 10, 2016, 10:44:21 PM10/10/16
to fds...@googlegroups.com

So what imwas wrong?

Find where the shared torque library is located and add that directory to your LD_LIBRARY_PATH


To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/fe9c92bc-34b7-4683-9e00-7233de8671fc%40googlegroups.com.

Daniel

unread,
Oct 10, 2016, 11:13:52 PM10/10/16
to FDS and Smokeview Discussions
I used another intel compiler and then reinstalled openmpi 1.8.4. 
I'm sorry, but you mean the "libtorque.so.2" file? It locates in the "FDS/FDS6/bin/LIB64" directory. I have added it to LD_LIBRARY_PATH and checked that.


在 2016年10月11日星期二 UTC+9上午11:44:21，Glenn Forney写道：
Reply all
Reply to author
Forward
0 new messages