Scatter Subarrays

[David T. Faux]

I am teaching myself the Python mpi4py module for programming in multiple processes. I have written the following piece of code to practice scatter.

    from mpi4py import MPI

    

    comm = MPI.COMM_WORLD

    size = comm.Get_size()

    rank = comm.Get_rank()

    

    if rank == 0:

       data = [i for i in range(8)]

    else:

       data = None

    data = comm.scatter(data, root=0)

    print str(rank) + ': ' + str(data)

Running the above code with 8 processes works great. However, when I run it with 4 processes, I get an error:

    Traceback (most recent call last):

      File "scatter.py", line 11, in <module>

        data = comm.scatter(data, root=0)

      File "Comm.pyx", line 874, in mpi4py.MPI.Comm.scatter (src/mpi4py.MPI.c:68023)

      File "pickled.pxi", line 656, in mpi4py.MPI.PyMPI_scatter (src/mpi4py.MPI.c:32402)

      File "pickled.pxi", line 127, in mpi4py.MPI._p_Pickle.dumpv (src/mpi4py.MPI.c:26813)

    ValueError: expecting 4 items, got 8

What does this error mean? My intention is to break up my large array of 8 items into small arrays of 8 / 4 = 2 items and send each process one such subarray. How do I do that? I would also like to generalize if possible to numbers of processes that do not divide evenly into 8 such as 3.

Thank you!

[Aron Ahmadia]

Hi Zeng,

This is what happens when tutorial code gets continuously reposted over the Internet, eventually it loses its original meaning!

This is an important piece of accompanying documentation to the mpi4py code:

Collective calls like scatter(), gather(), allgather(), alltoall() expect/return a sequence of Comm.size elements at the root or all process. They return a single value, a list of Comm.size elements, or None.

In other words, you always need to pass a sequence that is the same size as comm.

Well, what happens when you've got a sequence that isn't the same size as comm, and you want to divvy it up yourself?  

You could use this one-liner if the order doesn't matter:

data = range(10)

splitdata = [data[i::size] for i in range(size)]

If order matters, you need a slightly more complicated function:

def split_seq(seq, size):

        newseq = []

        splitsize = 1.0/size*len(seq)

        for i in range(size):

                newseq.append(seq[int(round(i*splitsize)):int(round((i+1)*splitsize))])

        return newseq

splitdata = split_seq(data,4)

Hope this helps!

-Aron

--
 
 

[Lisandro Dalcin]

This means that in order to scatter(), you need a sequence of EXACTLY
comm.size elements. If the sequence has more items, you need to build
yourself a list-of-lists from you data. mpi4py does not try to
second-guess users intentions.

> My intention is to break up my large array of 8
> items into small arrays of 8 / 4 = 2 items and send each process one such
> subarray. How do I do that? I would also like to generalize if possible to
> numbers of processes that do not divide evenly into 8 such as 3.
>

For example, you can use a balanced block-distrubution. This is a
pretty common approach in, for example, parallel linear algrebra
libraries and structured grid based PDE solvers.

def dist_block(L, size):
dist = [None] * size
i, N = 0, len(L)
for rank in range(size):
n = N // size + (N % size > rank)
dist[rank] = L[i:i+n]
i += n
return dist

# test
L = range(10)
for n in range(1,11):
print dist_block(L, n)

Other option is to use a balanced cyclic distribution. This trivial,
really a one-liner:

def dist_cyclic(L, size):
return [L[rank::size] for rank in xrange(size)]

L = range(10)
for n in range(1,11):
print n, dist_cyclic(L, n)

--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169