compile with mpi
robertkraus
this must be trivial, but I am not familair with mpi and I am tryign
this for the first time, now. My computer system has openmpi
installed, and the mpi compiler mpicc is sitting in
/usr/lib64/openmpi/1.4-gcc/bin/mpicc
When I run ./configure from the migrate-n source code distribution it
looks like it can't find mpi:
checking for MPI_Init in -lmpi... no
And here the output in full:
[kraus@XXX src]$ ./configure
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
svn: '.' is not a working copy
svn: '.' is not a working copy
checking for gmake... gmake
Linux
===============================
.
unknown
linux-gnu + + Linux + +
x86_64
===============================
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
+++++++++++++++++++++++++++++++++++++++
gcc
g++
-O3 -Wall -funroll-loops -fomit-frame-pointer -falign-loops=16 -
Dx86_64
+++++++++++++++++++++++++++++++++++++++
checking for a BSD-compatible install... /usr/bin/install -c
checking for pthread_create in -lpthread... yes
checking for MPI_Init in -lmpi... no
checking for exp in -lm... yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking for strings.h... (cached) yes
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking for an ANSI C-conforming const... yes
checking for size_t... yes
checking for int... yes
checking whether struct tm is in sys/time.h or time.h... time.h
checking for inline... inline
checking return type of signal handlers... void
checking for strftime... yes
checking for strcspn... yes
checking for strsep... yes
checking for lgamma... yes
checking which zlib implementation to use... autodetect
checking for main in -lz... yes
checking for gzbuffer... no
Copying source from Mersenne-Twister directory into main source
directory
configure: creating ./config.status
config.status: creating Makefile
config.status: creating conf.h
-----------------------------------------------------------------------
MIGRATE-N is now setup
to compile the program you need to type one of the
commands below (ordered by preference):
SINGLE CPU machines
make # with PDF output support
# this might fail on first attempt,
# if the make fails, please let me know
# often linking fails because the C++ pdf library
# and the core migrate parts (C) mismatch
# check the Makefile line for STDLIB= -lstdc++
SMP machines more than 1 CPU in machine
make thread # uses the thread library if present
make # on macs running macos 10.6+ using GrandCentral
PARALLEL Computers, Clusters with MPI software
make mpis # each cpu-core can be treated as a node
make mpis-thread # experimental, may increase load per node!
------------------------------------------------------------------------
SPEED and other TESTING
make single_short_test #this will run a short test and produce
#a single_short_test.log file showing
#the speed of your machine for a single
# CPU run, send this file to
bee...@fsu.edu
------------------------------------------------------------------------
When I make mpis then, it looks all nice but there is no mpi
executable built because mpicc is not found (/bin/sh: mpicc: command
not found).
Here, again, the full output:
[kraus@XXX src]$ make mpis
(cd ./haru ; gmake -f makefile.gcc "CC=g++" "CPP=gcc" "ARCH="
"MACOSVERSION=" "MYCPPFLAGS=-DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER ")
gmake[1]: Entering directory `/home/WUR/kraus003/migrate-3.2.7/src/
haru'
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfExceptions.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfList.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfUtils.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfStreams.cc
PdfStreams.cc: In member function âvirtual int
PdfMemStream::Write(const void*, int)â:
PdfStreams.cc:240: warning: dereferencing type-punned pointer will
break strict-aliasing rules
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfAutoPtrObjects.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfObjects.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfContents.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfPages.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfFonts.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfXref.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfDestinations.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfOutlines.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfBorderStyle.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfAnnotations.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfDoc.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfCatalog.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfInfo.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfEncoding.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfFontDef.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfXObjects.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfFontDef_Courier.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfFontDef_Helvetica.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfFontDef_Times.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfFontDef_Symbol.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfFontDef_ZapfDingbats.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfEncoding_ISO8859.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfEncoding_KOI8.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfEncoding_MSCP.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfMbFonts.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfCMap.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfUnicodeText.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfMbFontDef_Gothic.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfMbFontDef_Mincyo.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfMbFontDef_SimHei.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfMbFontDef_SimSun.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfMbFontDef_Ming.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfMbFontDef_Batang.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfMbFontDef_Dotum.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfCMap_RKSJ.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfCMap_EUC.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfCMap_GB_EUC.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfCMap_ETen_B5.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfCMap_KSCms_UHC.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfCMap_KSC_EUC.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfPngImage.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfJpegImage.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c PdfEncrypt.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c libharuc.cc
libharuc.cc: In function âint pdf_cid_type2_fontdef_ascent(void*)â:
libharuc.cc:164: warning: converting to non-pointer type âlong intâ
from NULL
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c libharuc_jpfonts.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c libharuc_krfonts.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c libharuc_cnsfonts.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c libharuc_cntfonts.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c libharu_png.cc
gcc -I. -Wall -O3 -DNDEBUG -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -c libharu_jpeg.cc
ar rc libharu.a PdfExceptions.o PdfList.o PdfUtils.o PdfStreams.o
PdfAutoPtrObjects.o PdfObjects.o PdfContents.o PdfPages.o PdfFonts.o
PdfXref.o PdfDestinations.o PdfOutlines.o PdfBorderStyle.o
PdfAnnotations.o PdfDoc.o PdfCatalog.o PdfInfo.o PdfEncoding.o
PdfFontDef.o PdfXObjects.o PdfFontDef_Courier.o PdfFontDef_Helvetica.o
PdfFontDef_Times.o PdfFontDef_Symbol.o PdfFontDef_ZapfDingbats.o
PdfEncoding_ISO8859.o PdfEncoding_KOI8.o PdfEncoding_MSCP.o
PdfMbFonts.o PdfCMap.o PdfUnicodeText.o PdfMbFontDef_Gothic.o
PdfMbFontDef_Mincyo.o PdfMbFontDef_SimHei.o PdfMbFontDef_SimSun.o
PdfMbFontDef_Ming.o PdfMbFontDef_Batang.o PdfMbFontDef_Dotum.o
PdfCMap_RKSJ.o PdfCMap_EUC.o PdfCMap_GB_EUC.o PdfCMap_ETen_B5.o
PdfCMap_KSCms_UHC.o PdfCMap_KSC_EUC.o PdfPngImage.o PdfJpegImage.o
PdfEncrypt.o libharuc.o libharuc_jpfonts.o libharuc_krfonts.o
libharuc_cnsfonts.o libharuc_cntfonts.o libharu_png.o libharu_jpeg.o
ranlib libharu.a
gmake[1]: Leaving directory `/home/WUR/kraus003/migrate-3.2.7/src/
haru'
Using system zlib architecture
make migrate-n-mpi "CC=mpicc" "CFLAGS = -O3 -Wall -funroll-loops -
fomit-frame-pointer -falign-loops=16 -Dx86_64 -DNEWVERSION -
DBFDEBUG -DNEXUSTREE -DHAVE_ZLIB -DZNZ -DHIGHBITS -DHAVE_CONFIG_H -
DHAVE_MALLOCWRAP -DMERSENNE_TWISTER -DMEXP=19937 -DMIGRATESUBVERSION=
\"\\\"\\\"\" -DMIGRATEVERSION=\"\\\"3.2.7\\\"\" -DNOJPEG -DNOPNG -
Iharu -DPRETTY -DLETTERPAPER -DMPI -DSLOWNET -DNEWREPLICANT -
DMPIREPLICANT" "LIBS = -Lharu -lharu -lstdc++ -lz -lm "
make[1]: Entering directory `/home/WUR/kraus003/migrate-3.2.7/src'
mpicc -O3 -Wall -funroll-loops -fomit-frame-pointer -falign-loops=16
-Dx86_64 -DNEWVERSION -DBFDEBUG -DNEXUSTREE -DHAVE_ZLIB -DZNZ -
DHIGHBITS -DHAVE_CONFIG_H -DHAVE_MALLOCWRAP -DMERSENNE_TWISTER -
DMEXP=19937 -DMIGRATESUBVERSION="\"\"" -
DMIGRATEVERSION="\"3.2.7\"" -DNOJPEG -DNOPNG -Iharu -DPRETTY -
DLETTERPAPER -DMPI -DSLOWNET -DNEWREPLICANT -DMPIREPLICANT -c aic.c
/bin/sh: mpicc: command not found
make[1]: *** [aic.o] Error 127
make[1]: Leaving directory `/home/WUR/kraus003/migrate-3.2.7/src'
make: *** [mpis] Error 2
How can I tell either ./configure or the Makefile where to find mpicc?
Cheers,
Robert
Peter Beerli
I guess that if you type in the shell
mpicc
it will return that it does not find that binary
For running and compiling migrate you need to make sure that
the PATH variable is set correctly
For example on our system we use this script (openmpi-gnu.sh)
and call it in our .bash_profile initialization to set the
PATH and LD_LIBRARY_PATH correctly.
Here the script:
#!/bin/bash
OPENMPI_DIR=/usr/mpi/gnu/openmpi
if [ $(id -u) -ge 500 ]; then
if ! echo ${PATH} | grep -q ${OPENMPI_DIR} ; then
export PATH=${OPENMPI_DIR}/bin:${PATH}
export LD_LIBRARY_PATH=${OPENMPI_DIR}/lib64:$LD_LIBRARY_PATH
fi
fi
you would need to adjust the path to the directory, that here is set to /usr/mmpi/gnu/openmpi
and then use
source openmpi-gnu.sh
in your .bash_profile
[if you use the bash shell, for csh or tcsh similar commands need to be used
after that (logout/login) mpicc should return something like "no file found " etc
and make mpis for migrate should work.
Peter
> --
> You received this message because you are subscribed to the Google Groups "migrate-support" group.
> To post to this group, send email to migrate...@googlegroups.com.
> To unsubscribe from this group, send email to migrate-suppo...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/migrate-support?hl=en.
>
Robert Kraus
Thanks for the pointer. Adapting your script didn't work, actually, but the pointer was about right. I think I solved it in a bit more simplistic way: I manually added the path to mpicc to my PATH variable in bash.profile, and defined the LD_LIBRARY_PATH variable also. That made it work!
I am facing the next problem now: The computer I wish to use is managed via SGE. I am compelte beginner with MPI and have no idea how SGE and MPI would work together. A search on google only turned up serious issues, but no "beginner's guide" or similar. Let me describe the system briefly:
A single server with 48 cores, SGE is installed, there is only one queue. I run migrate-n as:
mpirun -mca btl ^openib -np 10 ../src/migrate-n-mpi parmfile -nomenu
This funny mca and openib thing I found on google, because I had loads of error messages in my first trials because of some local network driver issues. That command sccessfully starts up a parallel migrate-n run. So, how can I pass this exact command into SGE? How will SGE know to assign 10 processors on 10 slots, and not 10 tasks to the same? What will happen if the queue is full? Will SGE wait until 10 slots are free and then launch my job? Or will it already start, as soon as one slot is free?
I can't ask the maintainer of the computer because he has never used MPI. It was just pre-installed if I understand him corectly.
Any help from someone who is running MPI migrate-n throguh SGE regularly?
Thanks,
Robert
-------- Original-Nachricht --------
> Datum: Fri, 25 Mar 2011 17:32:43 -0400
> Von: Peter Beerli <crece...@gmail.com>
> An: migrate...@googlegroups.com
> Betreff: Re: [migrate-support] compile with mpi
--
GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit
gratis Handy-Flat! http://portal.gmx.net/de/go/dsl
Peter Beerli
I assume that you can run a simpl SGE job. I use SGE on my own cluster and use the script at the end of the mail.
The submit script constructs a submit file like this and then submits it, in my case also to a single queue.
if you ask for more nodes than available the job will not start, so if you cluster has 48 nodes and you ask for 49 it will wait
forever, but if you as for say 10 48 node jobs each of them will be worked on one after another, with 12 nodes jobs, 3 of them will run concurrently
on the cluster. A submit file.sub file looks like this:
#!/bin/bash
#$ -m e
#$ -M userid@address
#$ -j y
#$ -S /bin/bash
#$ -l h_cpu=99:00:00
#$ -V
#$ -cwd
mpirun -np 23 /home/pbeerli/bin/migrate-n-mpi parmfile.testbayes -nomenu
and then use this command to start
qsub -pe mpi 23 file.sub
The script is a perl script, check whether perl is available at that /usr/bin/perl address, it is there in most unix systems.
copy/paste it into a file for example call it submit, put it into a directory that is in the PATH, chmod +x sumbit and try it out, it assumes that
migrate-n-mpi is also available through the PATH.
You use this:
submit program parameterfile jobname #nodes hour:minute:seconds
#!/usr/bin/perl -w
$dd = `date +\%F`;
@def=("migrate-n-mpi","parmfile",$dd,2,"99:00:00");
splice(@def,0,scalar(@ARGV),@ARGV);
if($def[0] eq "--help" || $def[0] eq "-h" || $def[0] eq "-h" || $def[0] eq "-v" || $def[0] eq "--h" || $def[0] eq "-help")
{
print "\nSyntax:\n\nsubmit program parameterfile jobname #nodes hour:minute:seconds\n\n";
exit;
}
$migrate = $def[0];
$parmfile = $def[1];
$dir = $def[2];
$node = $def[3];
$wallhour = $def[4];
$dir = "job".$dir.".sub";
#----------------------------------------------------------------
# creating job
open($file,">".$dir);
print $file "#!/bin/bash\n";
print $file "#\$ -m e\n";
#use something like this to notify you by email: print $file "#\$ -M userid\@yourinstititution\n";
print $file "#\$ -j y\n";
print $file "#\$ -S /bin/bash\n";
print $file "#\$ -l h_cpu=$wallhour \n";
print $file "#\$ -V\n";
print $file "#\$ -cwd\n";
print $file "mpirun -np $node $migrate $parmfile -nomenu\n";
close($file);
#----------------------------------------------------------------
# submitting job
print "qsub -pe mpi $node $dir (maxtime=$wallhour)\n";
`qsub -pe mpi $node $dir`;
#end of perl script
Robert Kraus
Cheers,
Robert
-------- Original-Nachricht --------
> Datum: Mon, 28 Mar 2011 09:18:28 -0400
--
NEU: FreePhone - kostenlos mobil telefonieren und surfen!
Jetzt informieren: http://www.gmx.net/de/go/freephone