automatic MPI datatype discovery for NumPy arrays and PEP-3118 buffers

[Lisandro Dalcin]

Hi folks, a few weeks ago I've pushed to SVN some support for
determining the appropriate MPI datatype for NumPy arrays (and builtin
array.array) and PEP-3118 buffers (though it is not being used too
much out there, e.g. AFAIK, NumPy still do not support it).

Only basic types are supported, i.e., all C-native integrals,
floating, and complex. For complexes, Fortran 77 datatypes are used if
the MPI implementation does not provides support for the C99 types
introduced this year in the new MPI-2.2 spec (note: MPICH2 1.2 already
have support for all new stuff in MPI-2.2). BTW, mpi4py SVN have
support for all the new stuff in MPI-2.2.

To see the "sugar", compare the two ways (first the new one, next the
old one) to Send()/Recv() below. A little improvement, but IMHO nice
enough for everyday programming to comment here.

from mpi4py import MPI
import numpy

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

# automatic MPI datatype discovery
# for NumPy arrays and PEP-3118 buffers
if rank == 0:
data = numpy.arange(100, dtype=numpy.float64)
comm.Send(data, dest=1, tag=13)
elif rank == 1:
data = numpy.empty(100, dtype=numpy.float64)
comm.Recv(data, source=0, tag=13)

# pass explicit MPI datatypes
if rank == 0:
data = numpy.arange(1000, dtype='i')
comm.Send([data, MPI.INT], dest=1, tag=77)
elif rank == 1:
data = numpy.empty(1000, dtype='i')
comm.Recv([data, MPI.INT], source=0, tag=77)



--
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

[Brian Granger]

Lisandro,

> Hi folks, a few weeks ago I've pushed to SVN some support for
> determining the appropriate MPI datatype for NumPy arrays (and builtin
> array.array) and PEP-3118 buffers (though it is not being used too
> much out there, e.g. AFAIK, NumPy still do not support it).
>
> Only basic types are supported, i.e., all C-native integrals,
> floating, and complex. For complexes, Fortran 77 datatypes are used if
> the MPI implementation does not provides support for the C99 types
> introduced this year in the new MPI-2.2 spec (note: MPICH2 1.2 already
> have support for all new stuff in MPI-2.2).  BTW, mpi4py SVN have
> support for all the new stuff in MPI-2.2.

This is great and I think it really makes sense to have this

> To see the "sugar", compare the two ways (first the new one, next the
> old one) to Send()/Recv() below. A little improvement, but IMHO nice
> enough   for everyday programming to comment here.

A small improvement, but an important one. Mainly because the new
call signatures have
a very Pythonic feel, which is nice. Great job!

Cheers,

Brian

> --
>
> You received this message because you are subscribed to the Google Groups "mpi4py" group.
> To post to this group, send email to mpi...@googlegroups.com.
> To unsubscribe from this group, send email to mpi4py+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mpi4py?hl=.
>
>
>



--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgra...@calpoly.edu
elli...@gmail.com