Shared memory for data structures and mpi4py.MPI.Win.Allocate_shared

[pierr...@gmail.com]

Hi all

I have a rather large numpy array that has to be loaded by all my MPI processes. I would like to know if, when several processes are on the same machine, they could share a portion of memory such that this array would be loaded only once per machine. So far, I understood that maybe this could be possible using Win_Allocate_Shared, but the only problem is that when using mpi4py, I have the following error

File "MPI/Win.pyx", line 106, in mpi4py.MPI.Win.Allocate_shared (src/mpi4py.MPI.c:126839)

NotImplementedError

Is this normal, or am I missing something in the compilation (I have openmpi 1.6.5)

Best and thanks a lot 

[Lisandro Dalcin]

On 20 April 2016 at 11:20, <pierr...@gmail.com> wrote:
> Hi all
>
> I have a rather large numpy array that has to be loaded by all my MPI
> processes. I would like to know if, when several processes are on the same
> machine, they could share a portion of memory such that this array would be
> loaded only once per machine. So far, I understood that maybe this could be
> possible using Win_Allocate_Shared,

Yes, that should work.

> but the only problem is that when using
> mpi4py, I have the following error
>
> File "MPI/Win.pyx", line 106, in mpi4py.MPI.Win.Allocate_shared
> (src/mpi4py.MPI.c:126839)
> NotImplementedError
>

You get that error simply because your MPI implementation does not
support MPI_Win_Allocate_Shared().

> Is this normal, or am I missing something in the compilation (I have openmpi
> 1.6.5)
>

I see. That Open MPI version is quite aged, please upgrade (likely to
1.10.2 to be sure).



--
Lisandro Dalcin
============
Research Scientist
Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)
Extreme Computing Research Center (ECRC)
King Abdullah University of Science and Technology (KAUST)
http://ecrc.kaust.edu.sa/

4700 King Abdullah University of Science and Technology
al-Khawarizmi Bldg (Bldg 1), Office # 0109
Thuwal 23955-6900, Kingdom of Saudi Arabia
http://www.kaust.edu.sa

Office Phone: +966 12 808-0459

[Pierre Yger]

I upgraded OpenMPI, and indeed, the function is now working. I just need to understand now how to write my numpy array into my shared memory

Thanks a lot for the feedback and the amazing work !

[Lisandro Dalcin]

On 20 April 2016 at 18:13, Pierre Yger <pierr...@gmail.com> wrote:
> I upgraded OpenMPI, and indeed, the function is now working. I just need to
> understand now how to write my numpy array into my shared memory
>

Here you have a trivial examples, run it in at least 2 processes.

from mpi4py import MPI
import numpy as np

comm = MPI.COMM_WORLD

size = 1000
itemsize = MPI.DOUBLE.Get_size()
if comm.Get_rank() == 0:
nbytes = size * itemsize
else:
nbytes = 0
win = MPI.Win.Allocate_shared(nbytes, itemsize, comm=comm)

buf, itemsize = win.Shared_query(0)
assert itemsize == MPI.DOUBLE.Get_size()
buf = np.array(buf, dtype='B', copy=False)
ary = np.ndarray(buffer=buf, dtype='d', shape=(size,))

if comm.rank == 1:
ary[:5] = np.arange(5)
comm.Barrier()
if comm.rank == 0:
print ary[:10]

> Thanks a lot for the feedback and the amazing work !
>
> Le mercredi 20 avril 2016 12:24:58 UTC+2, Pierre Yger a écrit :
>>
>> Hi all
>>
>> I have a rather large numpy array that has to be loaded by all my MPI
>> processes. I would like to know if, when several processes are on the same
>> machine, they could share a portion of memory such that this array would be
>> loaded only once per machine. So far, I understood that maybe this could be
>> possible using Win_Allocate_Shared, but the only problem is that when using
>> mpi4py, I have the following error
>>
>> File "MPI/Win.pyx", line 106, in mpi4py.MPI.Win.Allocate_shared
>> (src/mpi4py.MPI.c:126839)
>> NotImplementedError
>>
>> Is this normal, or am I missing something in the compilation (I have
>> openmpi 1.6.5)
>>
>> Best and thanks a lot
>

> --
> You received this message because you are subscribed to the Google Groups
> "mpi4py" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mpi4py+un...@googlegroups.com.
> To post to this group, send email to mpi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/mpi4py.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mpi4py/218c3f6f-28e4-4aed-a82a-ebcafc5175c2%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[Pierre Yger]

Again, thanks you so much! 

This is exactly what I was trying to do! Now I should be able to optimize considerably the memory footprint of my code

Best

Pierre

--
You received this message because you are subscribed to a topic in the Google Groups "mpi4py" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mpi4py/Fme1n9niNwQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mpi4py+un...@googlegroups.com.

To post to this group, send email to mpi...@googlegroups.com.
Visit this group at https://groups.google.com/group/mpi4py.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpi4py/CAEcYPwAtY-%3DEinegNn2vt0keTR3XRFQ1CpMg9Sqe%3DKk4Y5B1GA%40mail.gmail.com.