Running python C extension in mpi

[gorkiana]

I currently have a C library implemented in MPI. I intend to wrap it into python as a C extension, making it a module. 

My questions are:

1- Can I just make the wrappers and with that the python modules and just import and use them as any other module? Will they make use of mpi? Or do I need to load pypar, for instance?

2- If I load pypar and start mpi my mpi modules written in C will actually run in MPI?

Thank you for your help.

-artur palha

[Ole Nielsen]

Hi Artur

Thanks for your question. 

Your idea makes sense and I can't see any reason it wouldn't work.

>1- Can I just make the wrappers and with that the python modules and just import and use them as any other >module?

Yes you can. See the module mpiext.c for examples of how to make C code available to Python.

> Will they make use of mpi?

Yes, if they are linked against the MPI library and the program using them started with mpirun

> Or do I need to load pypar, for instance?

You would most likely need some of the functionality in pypar - such as the basic communication and getting your processor id for instance.

Alternatively, if your C code did all that, you could probably run without pypar. Pypar's purpose is to make MPI calls available to Python programs.

> 2. If I load pypar and start mpi my mpi modules written in C will actually run in MPI?

Yes that would be the idea. If you write them the same way mpiext.c is done and call them from a python program using pypar this should work.

But, of course, it can often be difficult to debug parallel programs...

Best thing to do is to read through the mpiext.c and pypar.py source code, run the pypar examples and think about what your modules will add.

Happy to discuss further

Cheers

Ole

[gorkiana maxima]

Hello Ole,

Thank you for your reply.

I started implementing things. Once I have it working I will tell the results.

Kind regards,

-artur palha

[Ole Nielsen]

Great - thanks

Good luck

Ole