Enzyme with MPI in C++ or Julia

[Daniel Abele]

Hi,

I'm looking into automatic differentiation for MPI programs. I saw your paper "Scalable Automatic Differentiation of Multiple Parallel Paradigms through Compiler Augmentation" (https://c.wsmoses.com/papers/enzymePar.pdf) and I had a few questions. Does the differentiation of MPI programs support reverse mode AD? It wasn't entirely clear to me from the paper, I may have missed it. Are there any operations that aren't supported (yet)? And since I have a choice of using C++ or Julia, does the Enzyme.jl package also support AD for MPI programs? I would think so, since it all works on LLVM IR, but I want to make sure before I start the work on my project. Anything I need to look out for in Julia as opposed to C++?

Kind regards,

Daniel