What is the right way to turn down the MPI environment?

[Liu Zhenhai]

Hi,
I recently use the mpi4py to parallelize my numpy-based optimization
program,it gain an significant speed up.
Thanks for providing such an amazing interface for MPI.
Now I run into a little problem .I use the MPI.COMM_WORLD.Abort()
function to stop the MPI environment.
But sometimes it doesn't work well .Not all python processes were
exited. the error says some process didn't finalize.
I have to kill the process one by one
So What is the right way to turn down the MPI environment?

[Aron Ahmadia]

You don't normally 'turn down' the MPI environment, but you can take a look at the cpi-dpm.py script (http://code.google.com/p/mpi4py/source/browse/demo/compute-pi/cpi-dpm.py) for an example of Dynamic Process management.  

A

--
You received this message because you are subscribed to the Google Groups "mpi4py" group.
To post to this group, send email to mpi...@googlegroups.com.
To unsubscribe from this group, send email to mpi4py+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mpi4py?hl=en.

[Lisandro Dalcin]

On 27 February 2012 09:32, Liu Zhenhai <1989...@gmail.com> wrote:
> Hi,
> I recently use the mpi4py to parallelize my numpy-based optimization
> program,it gain an significant speed up.
> Thanks for providing such an amazing interface for MPI.
> Now I run into a little problem .I use the MPI.COMM_WORLD.Abort()
> function to stop the MPI environment.

Abort() should be used as an attempt to forcefully abort a program in
case of an unrecoverable error state.

> But sometimes it doesn't work well .Not all python processes were
> exited. the error says some process didn't finalize.
> I have to kill the process one by one
> So What is the right way to turn down the MPI environment?
>

As Aron said, you don't normally need to "turn-down" MPI when using
mpi4py, as it takes care of finalizing MPI when the Python processes
are about to finish. Of course, you can call MPI.Finalize() yourself
to manually finish MPI. But that means that all your processes have
reached a point where they know no more work has to be done.

--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

[刘振海]

Hi,

Thanks for replying!

I am trying to learn more from the mpi4py demo. And at the same time I want to ask the follow question.

Here is the procedure of my program.

when the master find out the result has been converged,I want to turn down the MPI environment immediately, no matter the other worker have finished their job or not.What should I do? Have to call the Abort?

another plan: the master can send a special message to tell the other works to exit ,But I am not prefer to use this. since it is not efficient.

you have to wait for the worker to receive the special message.

Regards,

Liu Zhenhai

[Aron Ahmadia]

when the master find out the result has been converged,I want to turn down the MPI environment immediately, no matter the other worker have finished their job or not.What should I do? Have to call the Abort?

If you call Abort, this ends the program (usually ungracefully).  It seems like this is what you want the Master to do.

[Dag Sverre Seljebotn]

On 02/27/2012 09:28 PM, 刘振海 wrote:
> Hi,
> Thanks for replying!
> I am trying to learn more from the mpi4py demo. And at the same time I
> want to ask the follow question.

> Here is theprocedure of my program.

> when the master find out the result has been converged,I want to turn

> down the MPI environment immediately,no matter the other worker have

> finished their job or not.What should I do? Have to call the Abort?
> another plan: the master can send a special message to tell the other
> works to exit ,But I am not prefer to use this. since it is not efficient.
> you have to wait for the worker to receive the special message.

Yes, do this: Have master send a message to get workers to abort (and
the workers can do sys.exit(0) or just return from whatever they are
doing to end the program). Think about it: Some message must be sent
over the network one way or another to have the node stop the process,
right?

The real lesson though is that efficiency does not matter here. How many
messages do you send during the computation? How much communication
overhead (in percentage) is the shutdown message going to be?

Dag

>
> Regards,
> Liu Zhenhai
>
>
>
> 在 2012年2月28日 上午7:48，Lisandro Dalcin <dal...@gmail.com

> <mailto:dal...@gmail.com>>写道：

>
> On 27 February 2012 09:32, Liu Zhenhai <1989...@gmail.com

> Tel: +54-342-4511594 <tel:%2B54-342-4511594> (ext 1011)
> Tel/Fax: +54-342-4511169 <tel:%2B54-342-4511169>

>
> --
> You received this message because you are subscribed to the Google
> Groups "mpi4py" group.
> To post to this group, send email to mpi...@googlegroups.com

> <mailto:mpi...@googlegroups.com>.

> To unsubscribe from this group, send email to
> mpi4py+un...@googlegroups.com

> <mailto:mpi4py%2Bunsu...@googlegroups.com>.

[刘振海]

Hi,

The whole processes are not always exit when the master call Abort.Sometimes it does sometimes it doesn't.

It's indeed ungraceful.But I am ok with this.

The worker send messages to master based on probability in my algorithm.

I felt it's a little bit difficult to implement the algorithm with blocking function,

especial when to implenment the exit mechanism

Maybe I will use non-blocking function to rewrite.

Thank you guys!

Regards,

Liu Zhenhai

 

   <mailto:mpi4py@googlegroups.com>.

   To unsubscribe from this group, send email to

   mpi4py+unsubscribe@googlegroups.com

   <mailto:mpi4py%2Bunsubscribe@googlegroups.com>.

   For more options, visit this group at
   http://groups.google.com/group/mpi4py?hl=en.


--
You received this message because you are subscribed to the Google
Groups "mpi4py" group.
To post to this group, send email to mpi...@googlegroups.com.
To unsubscribe from this group, send email to

mpi4py+unsubscribe@googlegroups.com.

For more options, visit this group at
http://groups.google.com/group/mpi4py?hl=en.

--
You received this message because you are subscribed to the Google Groups "mpi4py" group.
To post to this group, send email to mpi...@googlegroups.com.

To unsubscribe from this group, send email to mpi4py+unsubscribe@googlegroups.com.