Compilation with gcc and openmpi on Linux Intel64 (no Intel Fortran)

392 views
Skip to first unread message

firefox

unread,
Feb 6, 2011, 9:46:53 PM2/6/11
to FDS and Smokeview Discussions
Hi everybody,

I am trying to compile FDS5 (parallel version) on Suse Linux 11.3 (64
bit) with openmpi and gcc (gcc-4_5-branch revision 160292), not using
the Intel Fortran compiler. I did manage to do that with FDS4 last
year and adapted the FDS5 makefile accordingly (see below).

The compilation works and I receive a new fds5_mpi.

When running I get

user@pc4:~/fds5/transrun/bla> mpirun -np 2 ../../bin/fds5_mpi t1.fds
mpirun noticed that job rank 0 with PID 5518 on node pc4.site exited
on signal 11 (Segmentation fault).
1 additional process aborted (not shown)

Would anyone know why there is this segmentation fault, could advise
on how to modify the makefile or suggest a different way to compile
FDS5 without using Intel Fortran compiler (if possible at all).

Regards,
firefox

-----------------------
# Makefile for Fire Dynamics Simulator (FDS), Version 5

# To use this makefile, select the appropriate compiler and OS from
the list below. For example, to
# compile the 64 bit parallel version (MPI) of FDS under Linux with
OpenMP using the Intel compilers type:
#
# make openmp_mpi_intel_linux_64

# A make utility for windows may be downloaded from
http://gnuwin32.sourceforge.net/packages/make.htm

# The variable VPATH defines where the source code is relative to the
current directory

VPATH = .

# Definition of the non-MPI or MPI object variables

# note: objf_serial and objf_mpi contain only Fortran routines ie not
isob.c

objf_serial = prec.o mpis.o smvv.o cons.o devc.o data.o type.o mesh.o
func.o irad.o \
ieva.o pois.o scrc.o radi.o evac.o part.o vege.o ctrl.o
turb.o dump.o hvac.o read.o \
mass.o wall.o fire.o divg.o init.o velo.o pres.o main.o
obj_serial = isob.o $(objf_serial)

objf_mpi = prec.o mpip.o smvv.o cons.o devc.o data.o type.o mesh.o
func.o irad.o \
ieva.o pois.o scrc.o radi.o evac.o part.o vege.o ctrl.o
turb.o dump.o hvac.o read.o \
mass.o wall.o fire.o divg.o init.o velo.o pres.o main.o
obj_mpi = isob.o $(objf_mpi)

# General Purpose Rules
no_target:
@echo \******** You did not specify a make target \********
@echo Please read the comments at the top of the makefile

setup:
%.o : %.mod

.SUFFIXES: .c .f90 .o .obj

# Files which have OpenMP compiler directives have to be compiled
# by using the FOPENMPFLAG
# read.f90 has to be compiled without optimization, otherwise
# errors occure when reading .fds files.

func.o: FFLAGS += $(FOPENMPFLAGS)
part.o: FFLAGS += $(FOPENMPFLAGS)
turb.o: FFLAGS += $(FOPENMPFLAGS)
mass.o: FFLAGS += $(FOPENMPFLAGS)
fire.o: FFLAGS += $(FOPENMPFLAGS)
divg.o: FFLAGS += $(FOPENMPFLAGS)
velo.o: FFLAGS += $(FOPENMPFLAGS)
pres.o: FFLAGS += $(FOPENMPFLAGS)
read.o: FOPTIMIZATIONFLAGS = -O0

.f90.o:
$(FCOMPL) -c $(FOPTIMIZATIONFLAGS) $(FFLAGS) $<
.f90.obj:
$(FCOMPL) -c $(FOPTIMIZATIONFLAGS) $(FFLAGS) $<
.c.o:
$(CCOMPL) -c $(CFLAGS) $<
.c.obj:
$(CCOMPL) -c $(CFLAGS) $<

#*** Compiler Specific Rules ***

all: INCLUDEDIR = /usr/lib64/mpi/gcc/openmpi/include
all: FFLAGS = -I $(INCLUDEDIR)
all: CFLAGS = -I $(INCLUDEDIR)
all: LFLAGS = -L /usr/lib64/mpi/gcc/openmpi/lib64 -lmpi -lmpi_f90 -
shared
all: obj = fds5_mpi
all: FCOMPL = gfortran
all: CCOMPL = gcc
all: FOPTIMIZATIONFLAGS = -O2
all: FOPENMPFLAGS = -fopenmp

all: setup $(obj_mpi)
$(FCOMPL) -o $(obj) $(FOPTIMIZATIONFLAGS) $(FFLAGS) $(LFLAGS) $
(FOPENMPFLAGS)
#*** End Compiler Specific Rules ***

# *** Object Dependencies ***

smvv.o : isob.o
cons.o : prec.o
mpis.o : prec.o
type.o : prec.o cons.o
devc.o : prec.o
pois.o : prec.o cons.o
mesh.o : prec.o type.o
data.o : prec.o func.o cons.o
func.o : prec.o cons.o type.o mesh.o devc.o
ctrl.o : prec.o cons.o type.o mesh.o func.o
turb.o : func.o prec.o cons.o mesh.o
scrc.o : prec.o cons.o mesh.o func.o pois.o
irad.o : func.o prec.o cons.o type.o mesh.o
ieva.o : func.o prec.o cons.o type.o mesh.o
fire.o : func.o prec.o cons.o type.o mesh.o
hvac.o : func.o prec.o cons.o type.o mesh.o ctrl.o
wall.o : func.o prec.o cons.o type.o mesh.o hvac.o
velo.o : func.o prec.o cons.o type.o mesh.o turb.o
evac.o : func.o prec.o cons.o type.o mesh.o ieva.o data.o
divg.o : func.o prec.o cons.o type.o mesh.o scrc.o evac.o
mass.o : func.o prec.o cons.o type.o mesh.o turb.o
radi.o : func.o prec.o cons.o type.o mesh.o irad.o
pres.o : func.o prec.o cons.o type.o mesh.o pois.o velo.o scrc.o
part.o : func.o prec.o cons.o type.o mesh.o devc.o data.o
vege.o : func.o prec.o cons.o type.o mesh.o devc.o part.o
read.o : func.o prec.o cons.o type.o mesh.o devc.o ctrl.o irad.o
evac.o hvac.o scrc.o data.o
init.o : func.o prec.o cons.o type.o mesh.o devc.o irad.o pois.o
data.o
dump.o : func.o prec.o cons.o type.o mesh.o devc.o evac.o smvv.o
isob.c turb.o scrc.o data.o
main.o : func.o prec.o cons.o type.o mesh.o devc.o smvv.o isob.c
mass.o divg.o velo.o wall.o fire.o irad.o ieva.o radi.o evac.o part.o
vege.o dump.o read.o init.o pres.o pois.o ctrl.o turb.o hvac.o scrc.o
data.o

#*** Clean Target to remove Object and Module files ***

.PHONY : clean
clean:
-rm -f *.o *.mod

Kevin

unread,
Feb 7, 2011, 7:57:24 AM2/7/11
to FDS and Smokeview Discussions
What kind of MPI are you using?

How many meshes does t1.fds have?

You might want to submit the job to the Issue Tracker so that we can
check that it runs on our system.
> # A make utility for windows may be downloaded fromhttp://gnuwin32.sourceforge.net/packages/make.htm

firefox

unread,
Feb 7, 2011, 3:14:12 PM2/7/11
to FDS and Smokeview Discussions
1) user@pc4:~/fds5> mpirun -version
mpirun (Open MPI) 1.2.8

2) t1.fds has 2 meshes

3) The case runs with the NIST provided executable fds5_mpi_linux_64.

4) During the compilation I encountered the following problems

4a) wall.f90:1398 has 133 characters --> deleted one blank
4b)
gfortran -c -O2 -I /usr/lib64/mpi/gcc/openmpi/include -fopenmp
fire.f90
fire.f90:117.103:

CORR,DYF,DELTA,Q_BOUND_1,Q_BOUND_2,Q_NEW,CRIT_FLAME_TMPA,YY_GET,DYAIR,H_F_0,H_F

1
Error: Symbol 'h_f_0' at (1) has no IMPLICIT type
fire.f90:117.109:

YF,DELTA,Q_BOUND_1,Q_BOUND_2,Q_NEW,CRIT_FLAME_TMPA,YY_GET,DYAIR,H_F_0,H_F_N,H_G

1
Error: Symbol 'h_f_n' at (1) has no IMPLICIT type
fire.f90:117.115:

TA,Q_BOUND_1,Q_BOUND_2,Q_NEW,CRIT_FLAME_TMPA,YY_GET,DYAIR,H_F_0,H_F_N,H_G_0,H_G

1
Error: Symbol 'h_g_0' at (1) has no IMPLICIT type
fire.f90:117.121:

OUND_1,Q_BOUND_2,Q_NEW,CRIT_FLAME_TMPA,YY_GET,DYAIR,H_F_0,H_F_N,H_G_0,H_G_N)
&

1
Error: Symbol 'h_g_n' at (1) has no IMPLICIT type
make: *** [fire.o] Fehler 1

--> added h_f_0, h_f_n, h_g_0 and h_g_n to REAL (EB) in line 59
(probably wrong data type?)

4c)
gfortran -c -O2 -I /usr/lib64/mpi/gcc/openmpi/include -fopenmp
divg.f90
divg.f90:424.59:

!$OMP DO COLLAPSE(3) PRIVATE(K,J,I,YY_GET,CP_MF)
1
Error: Symbol 'cp_mf' at (1) has no IMPLICIT type
divg.f90:231.47:

!$OMP DO COLLAPSE(3) PRIVATE(K,J,I,TMP_G,H_G,HDIFF)
1
Error: Symbol 'h_g' at (1) has no IMPLICIT type
make: *** [divg.o] Fehler 1

--> added cp_mf and h_g to REAL(EB) in line 33 (probably wrong data
type?)

Kevin

unread,
Feb 7, 2011, 3:35:35 PM2/7/11
to FDS and Smokeview Discussions
Update your repository and try again. I removed these unused variables
from the OpenMP lines. Normally, we do not compile using the OpenMP
directives. This is why we have not noticed these errors. Let me know
if you now compile "cleanly".

On Feb 7, 3:14 pm, firefox <ka4338-...@online.de> wrote:
> 1) user@pc4:~/fds5> mpirun -version
> mpirun (Open MPI) 1.2.8
>
> 2) t1.fds has 2 meshes
>
> 3) The case runs with the NIST provided executable fds5_mpi_linux_64.
>
> 4) During the compilation I encountered the following problems
>
> 4a) wall.f90:1398 has 133 characters --> deleted one blank
> 4b)
> gfortran -c -O2 -I /usr/lib64/mpi/gcc/openmpi/include -fopenmp
> fire.f90
> fire.f90:117.103:
>
> CORR,DYF,DELTA,Q_BOUND_1,Q_BOUND_2,Q_NEW,CRIT_FLAME_TMPA,YY_GET,DYAIR,H_F_0­,H_F
>
> 1
> Error: Symbol 'h_f_0' at (1) has no IMPLICIT type
> fire.f90:117.109:
>
> YF,DELTA,Q_BOUND_1,Q_BOUND_2,Q_NEW,CRIT_FLAME_TMPA,YY_GET,DYAIR,H_F_0,H_F_N­,H_G
>
> 1
> Error: Symbol 'h_f_n' at (1) has no IMPLICIT type
> fire.f90:117.115:
>
> TA,Q_BOUND_1,Q_BOUND_2,Q_NEW,CRIT_FLAME_TMPA,YY_GET,DYAIR,H_F_0,H_F_N,H_G_0­,H_G
>
> 1
> Error: Symbol 'h_g_0' at (1) has no IMPLICIT type
> fire.f90:117.121:
>
> OUND_1,Q_BOUND_2,Q_NEW,CRIT_FLAME_TMPA,YY_GET,DYAIR,H_F_0,H_F_N,H_G_0,H_G_N­)

firefox

unread,
Feb 7, 2011, 7:21:57 PM2/7/11
to FDS and Smokeview Discussions
1) The compilation works now, but the segmentation fault during
execution remains.
user@pc4:~/fds5/transrun/bla> mpirun -np 2 fds5_mpi t1.fds
mpirun noticed that job rank 0 with PID 5479 on node pc4.site exited
on signal 11 (Segmentation fault).
1 additional process aborted (not shown)

2) The same input file works with fds5_linux_64 and lammpi:

user@pc4:~/fds5> /usr/bin/mpirun -np 2 ../bin/fds5_linux_64 t1.fds

Fire Dynamics Simulator

Compilation Date : Fri, 29 Oct 2010

Version: 5.5.3; MPI Disabled; OpenMP Disabled
SVN Revision No. : 7031

Job TITLE : TransRUN, 20 MW, 3m/s
Job ID string : T1

3) Basically, what I want is to find a way to compile fds5 without
using the Intel Fortran compiler and, ideally, with OpenMPI. I will
probably need some guidance on some of the compiler and linker
switches in the makefile.

4) On x64 I could use fds5_linux_64. However, my intention is to get
the compilation with gcc and OpenMPI done on a x64 machine. I figure,
when I get this done, I can compile FDS5 on an IA64 machine (sgi Altix
with Itanium 2) running SLES 11, which is my actual aim.

Glenn Forney

unread,
Feb 7, 2011, 9:30:02 PM2/7/11
to FDS and Smokeview Discussions
Try recompiling after turning off optimizations. If that works (it
has worked for me before), then we'll try and figure out which routine
optimization is failing on.

Kevin

unread,
Feb 8, 2011, 8:11:46 AM2/8/11
to FDS and Smokeview Discussions
I also suggest that you dial back the FDS repository to SVN 7031 (you
only have to do this for the FDS_Source and FDS_Compilation
directories). We have not thoroughly debugged the most recent version
of FDS that you get when you do an SVN update. Then do as Glenn says,
turn down the optimization to -O2.
> > with Itanium 2) running SLES 11, which is my actual aim.- Hide quoted text -
>
> - Show quoted text -

Christian Rogsch

unread,
Feb 8, 2011, 8:26:11 AM2/8/11
to FDS and Smokeview Discussions
Sorry for not reading this post earlier.

The latest working version of FDS with OpenMP (not OpenMPI) is SVN
7052 (SVN 7051 is an update for the compiler directives, SVN 7052 has
an updated makefile).
Have a look at this issue:
http://code.google.com/p/fds-smv/issues/detail?id=1319#c2
I recommend to set your SVN to 7052 if you need OpenMP.
The OpenMP version will be updated this week. I do not update it every
day the OpenMP compiler directives because every change in the code
needs a new code check, and if there are a lot of changes during a
week, I can do checks the whole day ;-)
Regards,
Christian

firefox

unread,
Feb 8, 2011, 8:53:41 AM2/8/11
to FDS and Smokeview Discussions
@Glenn&Kevin: I used SVN Version 7681 and did turn optimization off
(had it a -O2 before) - still the same segmentation fault.

@Christian: I will probably wait for the updated version later this
week with the unused variables (see post above) removed.

Is it true that the Intel Fortran compiler is not necessary - it
probably creates faster code, does it?

Thanks a lot for your support!

Kevin

unread,
Feb 8, 2011, 9:16:45 AM2/8/11
to FDS and Smokeview Discussions
Try using the debug option when you compile. You can see examples in
the makefile in the repository. If you get a seg fault, the debugger
ought to tell you the line number where the problem occurs. If you
just have a seg fault, you must find a needle in a haystack. I also
suggest that you not try to compile with the OpenMP option until you
successfully compile with OpenMPI. There is a difference between
OpenMP and OpenMPI.

Christian Rogsch

unread,
Feb 8, 2011, 9:25:00 AM2/8/11
to FDS and Smokeview Discussions
Kevin is right.
Try first to compile FDS without OpenMP until you get a running
version, then you can try OpenMP.
You will find several entries about other compiler in the makefile,
maybe one is running for you.
And you are right, you do not need an Intel compiler, you can also use
any other compiler you want, but the flags are different comparing to
the intel compiler. And the question about faster code cannot be
answered in general, because if you read speed reports of different
compilers from different companies, each company has some examples
where their compiler is faster than other compilers. If you have a
system on which you want to run FDS, you have to download different
compilers (test versions) and compile FDS and make a speed test. If
you found the fastest one, you have still to ensure that fast ==
correct, because some aggressive optimization can lead to "wrong"
code... So that's a story with "open end"...

Christian Rogsch

unread,
Feb 13, 2011, 11:30:13 AM2/13/11
to FDS and Smokeview Discussions
@firefox: I've uploaded an updated openmp source code (SVN 7713). Feel
free to test.
Reply all
Reply to author
Forward
0 new messages