The proper way to configure for mpi is by using:
./configure --prefix=/home/local --with-mpi
It should automatically find your mpi compiler, in your case mpif90.
I think that if you use FC=ifort in combination with --with-mpi,
you'll create a mpi executable that is not linked against the proper
libraries.
Also after you switch between mpi and nonmpi builds, make sure you do
a "make clean".
Can you show me what error you get after:
make clean
./configure --prefix=/home/local --with-mpi
make
Kind regards,
Fedor
> --
> You received this message because you are subscribed to the Google Groups "XBeach" group.
> To post to this group, send email to xbe...@googlegroups.com.
> To unsubscribe from this group, send email to xbeach+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/xbeach?hl=en.
>
>
Hi ali,
Your libmpi cannot be found. It is probably under /opt/intel/ somwhere, in a lib directory. That path should be added to the LD_LIBRARY_PATH environment variable. There should be a script in the bin directory of the compiler that does that for you. On my system I think it's called ifortvars.sh. If you have that one you can put it in your .profile or .bashrc file like:
. path/to/ifortvars.sh #the dot is equivalent to the source command
or if you don't have an ifortvars.sh:
export LD_LIBRARY_PATH = $LD_LIBRARY_PATH:path/to/libdir
then restart your shell or source your profile and it should work.
Kind regards,
Fedor
I'll try and give a bit more information about what's going on in the
configure and make of xbeach + mpi. But I'll start with a short
summary on how you should proceed.
- revert all the changes (or better download a new copy)
- locate your mpif90 compiler (which mpif90)
- make sure the accompanying lib path is in your LD_LIBRARY_PATH, for
example if mpif90 is in /usr/local/bin/mpif90, then your
LD_LIBRARY_PATH should contain /usr/local/lib
- run ./configure --with-mpi
- run make clean
- run make
- run mpiexec -np 2 ./xbeach
If something gives an error, stop and mail the output. You can save
the output to a file by running a command with &> file.log after it,
for example ./configure &> config.log, saves all output, including
errors to a log file.
Now for some extra information about mpi and xbeach. The mpi version
of xbeach is a bit different from the normal version in 3 ways.
Different code is compiled, different libraries are linked and the
program is started differently.
I'll first explain how the different code is compiled.
At configuration time (when you run configure) you can decide whether
you want a mpi version or not, this is done with the option --with-mpi
. What happens when you give the option is defined in the file
configure.ac:
AC_ARG_WITH(mpi, [AS_HELP_STRING([--with-mpi],[Build with mpi support
@<:@default=check@:>@])], [],[])
This line defines the mpi option. When it is defined it will run the
ACX_MPI macro (in the m4 directory).
What this macro does is that it searches for all well known
mpi-fortran compilers. If it finds one, it will check to see if it
works and if so, use that as a compiler.
Another thing that happens is that it sets the variable USEMPI.
When you looked at the code you probably saw some lines starting with
#ifdef(USEMPI)
Those type of lines, starting with a #ifdef are, what's called,
preprocessor statements.
The typical order of compilation of languages like fortran and c/c++ is:
sourcecode -> [preprocessor] -> processed sourcecode -> [compiler] ->
object files (.o) -> [linker] -> executable
By using the preprocessor statements, we can skip some code for the
non-mpi version. For example in the mpi version when a variable
changes, it has to be broadcasted to the other computers/cpus where
xbeach is running. All specific mpi function calls are only compiled
when #ifdef(USEMPI) is true, meaning if USEMPI is defined.
One such line is the MPI_BCAST call. What that does is it sends over
an array to all other processes. It is explained in the mpi
documentation. You could not find that function in xbeach because it
is part of mpi, an external library. When you disabled it, you
basically stopped all communication between the xbeach nodes. That is
not a good idea :)
So to sum it up, by running ./configure --with-mpi, 2 things happen,
it finds the proper mpi capable compiler and generates mpi enabled
code using the USEMPI flag.
I'll now explain how the software is linked.
By defining the --with-mpi a new compiler is set. This compiler is
usually called mpif90. This is actually a normal fortran compiler with
some extra options set so you don't have to worry about linking and
compiling the mpi libraries. You can see this by running mpif90 -show.
For example on my machine it says:
f95 -Wl,-Bsymbolic-functions -I/usr/include/mpich2
-I/usr/include/mpich2 -L/usr/lib -L/usr/lib -lmpichf90 -Wl,-rpath
-Wl,/usr/lib -lmpichf90 -lmpich -lopa -lpthread -lrt
This means that it will find fortran modules in /usr/include/mpich2,
it will look for libraries in /usr/lib, it will link mpichf90, mpich,
opa, pthreat and rt. It actually does a bit more, you can see this by
looking in mpif90, it's usually a bash script wrapping your normal
compiler.
So by using the option --with-mpi, we get a compiler which knows how
to link to the proper mpi modules. Now if you run configure with the
option FC=g95, FC=gfortran or FC=ifort, what you're doing is replacing
the line with all the linking options which are needed to create a
working mpi executable by just the name of the compiler. This causes
the problem during compilation that it can't find "use mpi", and
during linking that it can't find references to mpi_bcast and other
mpi functions. Therefore it's best to use a proper mpi fortran
compiler, in most cases mpif90. If it doesn't find it, because it's
for example named openmpif90, you can set it manually using
MPIFC=openmpif90 ./configure --with-mpi. This way it knows that it
should use that as a mpi compiler. Again using
MPIFC=ifort,g95,gfortran is not a good idea.
In your case you used FC=g95, had to strip out mpi specific lines like
xmaster, mpi_bcast, giving you a half mpi/half non mpi version, that
probably doesn't run anymore.
To summarize, --with-mpi makes sure that the mpif90 compiler is found.
Mpif90 knows how to create a mpi executable, the normal compilers
don't, so always use mpif90 to make the mpi version.
The final thing: How to run an mpi version of xbeach, compiled with
USEMPI, and linked against the mpi libraries (see ldd ./xbeach).
To tun the mpi enabled xbeach version, you run the mpiexec command
followed by the executable:
mpiexec -np 2 ./xbeach
This mpiexec program makes sure that the programs (xbeach is started
multiple times).
If at this step .so object files from mpi are not found, then your
system is not properly configured to run mpi executables. You probably
need to setup the LD_LIBRARY_PATH to point to the mpi libraries. But
if you installed mpi using yum/yast or apt it usually just works.
I hope this makes it a bit clearer. I wrote it down a bit detailed, so
I can copy this text into the manual. You can find more information on
mpi and the autotools build information on the web.
Please try the suggestions on the start of the email and send the
output if it fails.
Kind regards,
Fedor
So you get one warning and a few errors:
I'll try and explain what's happening. The third error (Error3 below)
is important, the others are just interesting.
Warning1:
mpif90 -g -o makeincludes makeincludes.o
/opt/intel/fce/9.1.039/lib/libimf.so: warning: warning: feupdateenv is
not implemented and will always fail
Explanation: According to the release notes of the intel 9 compiler,
this is nothing to worry about. From the release notes: "This warning
is due to a mismatch of library types and can be ignored. The warning
will not appear if -i-dynamic is used."
Error1:
configure:2436: ifort -qversion >&5
ifort: Command line warning: ignoring option '-q'; no argument
required
ifort: Command line error: no files specified; for help type "ifort -
help"
Explanation: The configure script is testing your compiler, in this
case it is testing if the compiler can accept a -qversion argument.
The ibm compiler accepts that option and displays the version and
release of the compiler being invoked. The intel fortran compiler
doesn't use that option. Nothing to worry about, it will find the -v
option to display the version. The configure script tries to see how
your compiler works by just running it, this has proven to be a robust
method to be able to support the many hundreds of different compilers
that are available.
Error2:
configure:2689: checking whether we are using the GNU Fortran compiler
configure:2702: ifort -c conftest.F >&5
fortcom: Error: conftest.F, line 3: Syntax error, found END-OF-
STATEMENT when expecting one of: => = . ( : %
choke me
---------------^
fortcom: Error: conftest.F, line 3: This statement is positioned
incorrectly and/or has syntax errors.
choke me
---------------^
compilation aborted for conftest.F (code 1)
Explanation: This is also nothing to worry about, the configure script
is trying to compile a program to see if it is the gnu compiler. It
isn't so it should choke on the line choke me :). You can see that
these errors are nothing to worry about because the configure script
just keeps running, it says: "checking whether we are using the GNU
Fortran compiler: No" and continues.
Error3:
mpif90 -DHAVE_CONFIG_H -I. -DUSEMPI -DHAVE_MPI_WTIME -g -c -o
xbeach-params.o `test -f 'params.F90' || echo './'`params.F90
fortcom: Error: params.F90, line 1024: There is no matching specific
subroutine for this generic subroutine call. [MPI_BCAST]
call MPI_Bcast(par,sizeof(par),MPI_BYTE,xmpi_master,xmpi_comm,ierror)
-----^
compilation aborted for params.F90 (code 1)
Explanation: Now this is a problem. Even though you are using the
mpif90 compiler it can't find the function MPI_BCAST. This is a bit
strange because it could find the function MPI_INIT. You can see this
in the lines:
configure:2832: checking for MPI_Init
.....
configure:2841: result: yes
The MPI_BCAST function is in mpi.mod, located in the include directory
next to your mpif90 compiler. We already know it can find it because
that was tested by the configure script (it found MPI_INIT). So what
happened is that someone (being me) forgot to "use" the mpi module.
It's a bit strange because it did compile on gfortran and on the intel
10 and 11 compiler. I'll fix this issue and test it and upload it. In
the meantime you can add a line to the distribute_par subroutine (at
line 1024 in params.F90), like this:
#ifdef USEMPI
subroutine distribute_par(par)
use mpi ! <<<< I forgot this, add this for now..
use xmpi_module
That should solve it. I hope this helps. Thanks for your persistence :)
Kind regards,
Fedor Baart
About this line:
character(len=20), dimension(:), allocatable :: dimensions ! the
I used an allocatable array inside a type. This is part of the fortran
2003 specification, which is not supported by some compilers (g95 +
gfortran < 4.2).
I think for older compiler it only works like this:
character(len=20), dimension(:), pointer :: dimensions
But I'll probably change it to:
character(len=20), dimension(maxndimensions) :: dimensions
I'll try and rewrite that part, so it also works on older compilers. A
way around this would be to use a more up to date compiler. Can you
check what compiler you use on that machine? You can find that out by
mpif90 -show.
Kind regards,
Fedor
vars
1
Error: ALLOCATABLE attribute conflicts with DUMMY attribute at (1)
In file params.F90:1197
Error: Function 'sizeof' at (1) has no IMPLICIT type
So as it failed I did the second one, " character(len=20),
dimension(maxndimensions) :: dimensions "
and then "./configure --with-mpi --prefix=/home/ali MPIFC=mpif90"
the output was like below:
character(len=20), dimension(maxndimensions) :: dimensions ! the
dimensions o
1
Error: Component 'dimensions' of 'arraytype' at (1) must have constant
I think you are right that the issue is with the old compiler
probably.
And for the compiler version, I did "mpif90 -show" and it said like:
"f95 -I/usr/local/include -I/usr/local/include -L/usr/local/lib -L/usr/
local/lib -lmpichf90 -lmpichf90 -lmpich -lopa -lpthread -lrt
The other server that you helped me compile the model looks to work
properly, but as I tried the ZWIN testcase in parallel, there appeared
an error saying:
" Error: wave stationary solver not compatable with MPI "
Were there any other specific input supposed to be ?
The gfortran 4.1 is a bit too old I'm afraid. It probably won't work.
Although I did rewrite some of it to make it more compatible (see
latest changes in the trunk) it probably won't solve your problems.
The sizeof function for example is an extension that is not available
for gfortran 4.1. Your best bet for this machine is to update your
compiler. For most systems updated compilers are available. You might
even have one installed under for example gfortran43 or something.
Otherwise you can get a newer gfortran compiler from the gcc website
http://gcc.gnu.org/ . Besides a new compiler you'll also need to build
or install an accompanying mpi, otherwise the mpi.mod (the mpi fortran
module) won't be compatible. I've tested xbeach with openmpi and
mpich2.
Kind regards,
Fedor
Did you also update your mpi? It is probably using your old compiler.
You can get openmpi here:
http://www.open-mpi.org/software/ompi/v1.4/
Make sure you build openmpi with your new compiler, something like:
FC=gfortran-4.5 F77=gfortran-4.5 ./configure
make
make install
What OS are you using, btw?
Kind regards,
Fedor