parallel compilation error, mpi type mismach

[Ilja Maljutenko]

Dear all,

After updating the getm-git version I stucked into weird problem. 

 The boundaries of parallel sub-domains created mess in all variables ( sealevel elevation + - 6 m, 0 salinity , etc ...)

I believe that following compiling error is to blame ...

mpif90 -DIFORT -DFORTRAN95 -DREAL_4B=real\(kind=4\) -DNETCDF3 -DGETM_PARALLEL -DPRODUCTION  -O3 -i-static -mp1  -module /home/ilja.maljutenko/GETM/getm-git/modules/IFORT -I/home/ilja.maljutenko/GETM/getm-git/include -I/home/ilja.maljutenko/GETM/getm-git/modules/IFORT -I/home/ilja.maljutenko/GOTM/gotm-git_cpfeb/modules/IFORT -I/home/ilja.maljutenko/netcdf4/include -w95  -c halo_mpi.F90

halo_mpi.F90(622): warning #6075: The data type of the actual argument does not match the definition.   [SIZEOF_REALTYPE]

   call MPI_TYPE_EXTENT(MPI_DOUBLE_PRECISION,sizeof_realtype,ierr)

---------------------------------------------^

halo_mpi.F90(629): warning #6075: The data type of the actual argument does not match the definition.

   call MPI_TYPE_HVECTOR(2,1,x_size*sizeof_realtype,x_line,x_lines,ierr)

-----------------------------------^

halo_mpi.F90(637): warning #6075: The data type of the actual argument does not match the definition.

   call MPI_TYPE_HVECTOR(2,1,1*sizeof_realtype,y_line,y_lines,ierr)

------------------------------^

halo_mpi.F90(645): warning #6075: The data type of the actual argument does not match the definition.

   call MPI_TYPE_HVECTOR(2,1,x_size*sizeof_realtype,halo_line,halo_square,ierr)

-----------------------------------^

halo_mpi.F90(649): warning #6075: The data type of the actual argument does not match the definition.

   call MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,x_line,xz_slice,ierr)

------------------------------------^

halo_mpi.F90(653): warning #6075: The data type of the actual argument does not match the definition.

   call MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,x_lines,xz_slices,ierr)

------------------------------------^

halo_mpi.F90(658): warning #6075: The data type of the actual argument does not match the definition.

   call MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,y_line,yz_slice,ierr)

------------------------------------^

halo_mpi.F90(662): warning #6075: The data type of the actual argument does not match the definition.

   call MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,y_lines,yz_slices,ierr)

------------------------------------^

halo_mpi.F90(685): warning #6075: The data type of the actual argument does not match the definition.

   call MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,halo_square, &

------------------------------------^

ar rv /home/ilja.maljutenko/GETM/getm-git/lib/IFORT/libfutils_prod.a halo_mpi.o

a - halo_mpi.o

With using mpif90 as

mpif90 -show

ifort -I/opt/intel-old/mpich2/include -I/opt/intel-old/mpich2/include -L/opt/intel-old/mpich2/lib -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt -lpthread

The error did not stop compilation and let me to run the model until error appeared in output.

Such mismatch did not appear when i tried to compile getm-1.8.0  source from my backup folder (source in tar here http://tiny.cc/bup_getmv180 ), but reappeared with previous versions (origin, v1_6_0) that i downloaded from git://getm.git.sourceforge.net. So i wonder if there have been some global changes for all the versions meanwhile? 

I think I've also excluded the source of error from environmental variables (not sure tho). But if they are needed i can provide them too.

Any help would be greatly appreciated,

BR,

--
~ Ilja 

[Knut]

Hi Ilja,

although this problem seems to be related to MPI... :-)
(have you checked your mpich settings?)
... my first question is: Do you compile with -DSTATIC?

Knut

> Dear all,
>
> After updating the getm-git version I stucked into weird problem.
> The boundaries of parallel sub-domains created mess in all variables
> ( sealevel elevation + - 6 m, 0 salinity , etc ...)
> I believe that following compiling error is to blame ...
>

> /mpif90 -DIFORT -DFORTRAN95 -DREAL_4B=real\(kind=4\) -DNETCDF3

> -DGETM_PARALLEL -DPRODUCTION -O3 -i-static -mp1 -module
> /home/ilja.maljutenko/GETM/getm-git/modules/IFORT
> -I/home/ilja.maljutenko/GETM/getm-git/include
> -I/home/ilja.maljutenko/GETM/getm-git/modules/IFORT
> -I/home/ilja.maljutenko/GOTM/gotm-git_cpfeb/modules/IFORT

> -I/home/ilja.maljutenko/netcdf4/include -w95 -c halo_mpi.F90/
> /halo_mpi.F90(622): warning #6075: The data type of the actual
> argument does not match the definition. [SIZEOF_REALTYPE]/
> / call MPI_TYPE_EXTENT(MPI_DOUBLE_PRECISION,sizeof_realtype,ierr)/
> /---------------------------------------------^/
> /halo_mpi.F90(629): warning #6075: The data type of the actual
> argument does not match the definition./
> / call
> MPI_TYPE_HVECTOR(2,1,x_size*sizeof_realtype,x_line,x_lines,ierr)/
> /-----------------------------------^/
> /halo_mpi.F90(637): warning #6075: The data type of the actual
> argument does not match the definition./
> / call MPI_TYPE_HVECTOR(2,1,1*sizeof_realtype,y_line,y_lines,ierr)/
> /------------------------------^/
> /halo_mpi.F90(645): warning #6075: The data type of the actual
> argument does not match the definition./
> / call
> MPI_TYPE_HVECTOR(2,1,x_size*sizeof_realtype,halo_line,halo_square,ierr)/
> /-----------------------------------^/
> /halo_mpi.F90(649): warning #6075: The data type of the actual
> argument does not match the definition./
> / call
> MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,x_line,xz_slice,ierr)/
> /------------------------------------^/
> /halo_mpi.F90(653): warning #6075: The data type of the actual
> argument does not match the definition./
> / call
> MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,x_lines,xz_slices,ierr)/
> /------------------------------------^/
> /halo_mpi.F90(658): warning #6075: The data type of the actual
> argument does not match the definition./
> / call
> MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,y_line,yz_slice,ierr)/
> /------------------------------------^/
> /halo_mpi.F90(662): warning #6075: The data type of the actual
> argument does not match the definition./
> / call
> MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,y_lines,yz_slices,ierr)/
> /------------------------------------^/
> /halo_mpi.F90(685): warning #6075: The data type of the actual
> argument does not match the definition./
> / call MPI_TYPE_HVECTOR(o,1,xy_size*sizeof_realtype,halo_square, &/
> /------------------------------------^/
> /ar rv
> /home/ilja.maljutenko/GETM/getm-git/lib/IFORT/libfutils_prod.a
> halo_mpi.o/
> /a - halo_mpi.o/
> /
> /
>
> With using mpif90 as
>
> /
> mpif90 -show
> //

> ifort -I/opt/intel-old/mpich2/include
> -I/opt/intel-old/mpich2/include -L/opt/intel-old/mpich2/lib
> -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt -lpthread

> /

>
>
> The error did not stop compilation and let me to run the model until
> error appeared in output.
> Such mismatch did not appear when i tried to compile getm-1.8.0
> source from my backup folder (source in tar here
> http://tiny.cc/bup_getmv180 ), but reappeared with previous versions
> (origin, v1_6_0) that i downloaded from git://getm.git.sourceforge.net

> <http://getm.git.sourceforge.net>. So i wonder if there have been some

[Karsten Bolding]

Hello

On Thu, Mar 22, 2012 at 05:08:47 +0200, Ilja Maljutenko wrote:
> Dear all,

<snip>

Is it an completely un-modified version of the code?

version of MPICH2?

just after line 622 in halo_mpi.F90 make a print statement of
sizeof_realtype. MPI_REALTYPE expands to MPI_DOUBLE_PRECISION as it
should.

>
> BR,
> --
> ~ Ilja

Karsten

--
http://www.getm.eu

[Ilja Maljutenko]

Hi,

On Thu, Mar 22, 2012 at 5:25 PM, Knut <knut...@gmx.net> wrote:

Hi Ilja,

although this problem seems to be related to MPI... :-)
(have you checked your mpich settings?)

Yes i did, but I couldn't blame anything there, since with same settings everything seems to work when running setup with old source.

 >> mpich2version

MPICH2 Version:         1.4.1p1

MPICH2 Release date:    Thu Sep  1 13:53:02 CDT 2011

MPICH2 Device:          ch3:nemesis

MPICH2 configure:       CC=icc CXX=icpc F77=ifort FC=ifort --prefix=/opt/intel-old/mpich2 --with-pm=mpd:gforker --with-device=ch3:nemesis --enable-threads=default --with-thread-package=posix --enable-mpe --enable-nmpi-as-mpi --with-mpe

MPICH2 CC:      icc    -O2

MPICH2 CXX:     icpc   -O2

MPICH2 F77:     ifort   -O2

MPICH2 FC:      ifort   -O2

 

... my first question is: Do you compile with -DSTATIC?

Yes, if I compile the setup -DSTATIC is defined. 

(The error i copied appeared when I compiled in getm directory)

On Thu, Mar 22, 2012 at 5:38 PM, Karsten Bolding <kar...@bolding-burchard.com> wrote:

Hello

On Thu, Mar 22, 2012 at 05:08:47 +0200, Ilja Maljutenko wrote:
> Dear all,

<snip>

Is it an completely un-modified version of the code?

The git-version that produced that error is compleatly un-modified. 

Not sure for the v1.8.0. But i'm pretty sure that everything related to parallelism or types in cppdefs.h is untouched there also.

version of MPICH2?

 

I belive it is this 
MPICH2 Version:         1.4.1p1

 

just after line 622 in halo_mpi.F90 make a print statement of
sizeof_realtype. MPI_REALTYPE expands to MPI_DOUBLE_PRECISION as it
should.

 

The print statmenet is:

    ~I output:

         sizeof_realtype =            0

--
~ Ilja 

[Ilja Maljutenko]

On Thu, Mar 22, 2012 at 6:21 PM, Ilja Maljutenko <ilj...@gmail.com> wrote:

Hi,

<snip> 

just after line 622 in halo_mpi.F90 make a print statement of
sizeof_realtype. MPI_REALTYPE expands to MPI_DOUBLE_PRECISION as it
should.

 

The print statmenet is:

    ~I output:

         sizeof_realtype =            0

Without the error it was : 

    ~I output:

         sizeof_realtype =            8

:/

--
~ Ilja 

[Karsten Bolding]

On Thu, Mar 22, 2012 at 06:21:28 +0200, Ilja Maljutenko wrote:
> Hi,

<snip>

>
> The print statmenet is:
> ~I output:
> sizeof_realtype = 0
>

this is surely wrong as it gives the number of bytes taken by the chosen
MPI_REALTYPE

>
> --
> ~ Ilja

--
http://www.getm.eu

[Karsten Bolding]

with the present git version I get:

call MPI_TYPE_EXTENT(MPI_REALTYPE,sizeof_realtype,ierr)

+STDERR 'kurt ',sizeof_realtype

kb@orca:/data/kb/getm-setups/git/box_cartesian$ grep kurt *.stderr
box_cartesian.000.stderr: kurt 8
box_cartesian.001.stderr: kurt 8
box_cartesian.002.stderr: kurt 8
box_cartesian.003.stderr: kurt 8

and no problems with results

so sorry I can't be of any help.

kb@orca:/data/kb/getm-setups/git/box_cartesian$ mpich2version

MPICH2 Version: 1.4.1p1
MPICH2 Release date: Thu Sep 1 13:53:02 CDT 2011
MPICH2 Device: ch3:nemesis

MPICH2 configure: -prefix=/opt/mpich2-1.4.1p1
MPICH2 CC: gcc -O2
MPICH2 CXX: c++ -O2

MPICH2 F77: ifort -O2
MPICH2 FC: ifort -O2

Karsten

--
http://www.getm.eu

[Ilja Maljutenko]

I temporarily solved it by simply setting sizeof_realtype = 8  after the call MPI_TYPE_EXTENT(.. line.

That seemed to work well enough to make the run.

If i manage to workout this issue I will come back to this thread.

Thanks for your efforts!

[Knut]

Hi Ilja,
>
>
> I temporarily solved it by simply setting /sizeof_realtype = 8/ after
> the /call MPI_TYPE_EXTENT(.. /line.

good.

> That seemed to work well enough to make the run.
> If i manage to workout this issue I will come back to this thread.

Once I had the same error but I only remember that there was something
very trivial that fixed it (either -DSTATIC, or simply a distclean, at
least nothing fancy...). Maybe you can figure it out :-)

Knut

>
> Thanks for your efforts!
>
>
> On Thu, Mar 22, 2012 at 7:49 PM, Karsten Bolding

> <kar...@bolding-burchard.com <mailto:kar...@bolding-burchard.com>>

[Karsten Bolding]

Hello

On Thu, Mar 22, 2012 at 05:08:47 +0200, Ilja Maljutenko wrote:
> Dear all,

<snip>

> Any help would be greatly appreciated,

I've made an attempt to fix the problem. I had the same problem on a 64
bit machine where I also got sizeof_realtype=0 instead of 8.

When I introduce:
INTEGER(KIND=MPI_ADDRESS_KIND) :: sizeof_realtype
instead of just:
integer :: sizeof_realtype

I get correct results.

Commit to devel branch - please somebody check on 32 bit machine.

This should likely be back-ported to stable.

Another swing on the issue is that MPI_TYPE_EXTENT is deprecated and
should be replaced by MPI_TYPE_GET_EXTENT - not done yet.

[Ilja Maljutenko]

Hello 

On Wed, Apr 4, 2012 at 2:43 PM, Karsten Bolding <kar...@bolding-burchard.com> wrote:

Hello

On Thu, Mar 22, 2012 at 05:08:47 +0200, Ilja Maljutenko wrote:
> Dear all,

<snip>

> Any help would be greatly appreciated,

I've made an attempt to fix the problem. I had the same problem on a 64
bit machine where I also got sizeof_realtype=0 instead of 8.

When I introduce:
  INTEGER(KIND=MPI_ADDRESS_KIND) :: sizeof_realtype
instead of just:
  integer :: sizeof_realtype

I get correct results.

<snip> 

 

It sure did fix the issue here :)

I downloaded the getm-git and compiled with no errors. Also the results were correct. 

I have here 64bit machine, so i can't test it on 32bit.

Thanks!

--
~ Ilja 

[fatemeh chegini]

Dear Karsten,

I got IIja's error while compiling getm with mpi on centos 64bit. I had to remove your commit INTEGER(KIND=MPI_ADDRESS_

KIND) :: sizeof_realtype and use  integer :: sizeof_realtype again to make it work

Best regards,

Fateme

[Karsten Bolding]

I'm confused - is it the new commit that creates problems? Because here they should be declared as integer in the default compilation.

--
 
---
You received this message because you are subscribed to the Google Groups "GETM-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to getm-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[fatemeh chegini]

yes. The new commit creates problems for me because when I remove the
(KIND=MPI_ADDRESS_KIND) part it works fine.

On 6/11/13, Karsten Bolding <kar...@bolding-burchard.com> wrote:
> I'm confused - is it the new commit that creates problems? Because here
> they should be declared as integer in the default compilation.
>
>
> On Tue, Jun 11, 2013 at 10:19 AM, fatemeh chegini
> <fatem...@gmail.com>wrote:
>
>> Dear Karsten,
>>
>> I got IIja's error while compiling getm with mpi on centos 64bit. I had
>> to
>> remove your commit INTEGER(KIND=MPI_ADDRESS_
>>

>> **KIND) :: sizeof_realtype and use integer :: sizeof_realtype again to

>> make it work
>>
>> Best regards,
>>
>> Fateme
>>
>>
>> On Wednesday, April 4, 2012 4:13:32 PM UTC+4:30, Karsten Bolding wrote:
>>>
>>> Hello
>>>
>>> On Thu, Mar 22, 2012 at 05:08:47 +0200, Ilja Maljutenko wrote:
>>> > Dear all,
>>>
>>> <snip>
>>>
>>> > Any help would be greatly appreciated,
>>>
>>> I've made an attempt to fix the problem. I had the same problem on a 64
>>> bit machine where I also got sizeof_realtype=0 instead of 8.
>>>
>>> When I introduce:

>>> INTEGER(KIND=MPI_ADDRESS_**KIND) :: sizeof_realtype

[Karsten Bolding]

but KIND=MPI_ADDRESS_KIND is not used at all in the default compilation. Have a look in halo_mpi.F90 and search for _MPI_TYPE_EXTENT_.

Are you sure you have made a 'git pull'?

[fatemeh chegini]

I have getm devel 2.5 which I pulled by git

in line 604 of halo_mpi.f90 I have
INTEGER(KIND=MPI_ADDRESS_KIND) :: sizeof_realtype

is this not used?

[Karsten Bolding]

Mu guess is that you have not updated the SF repository as described in a mail to getm-users on Feb. 26th.

I at least have something very different in line 604.

[fatemeh chegini]

Yes you're right. I used the SF indicated in the GETM website. It
works fine now. Thank you.