Using a multiprocessing pool with MDA

[Tyler Reddy]

Hi,

I've been able to use a multiprocessing Pool strategy with asynchronous evaluation of 32 MDA tasks in parallel. While this is quite powerful, as the code can continuously feed in new MDA tasks as soon as a processor is freed up, there is one annoying problem which has nothing to do with MDA: exceptions raised by the pool worker processes are not propagated back up to the parent.

This is apparently fixed in Python 3.4. There's some discussion here: http://stackoverflow.com/questions/6728236/exception-thrown-in-multiprocessing-pool-not-detected

Anyways, this is the first instance I've observed where a Python 3 fix is desirable for my MDA workflow. I don't really want to use Python 3 anytime soon, but I'm just making this info available (I think there's already an issue open for eventually porting MDA to Python 3--it might be sensible to add this to that thread).

Cheers,
Tyler