reusing cython files with mcsolve?

[Paul Haljan]

Is it possible to reuse the compiled cython function files with mcsolve called from separate processors?

I am attempting to use mcsolve on a cluster with a separate processor for each parameter run of a t-dependent Hamiltonian with collapse operators (qutip v3.0.1). The call to mcsolve often fails during its cython compilation of the string-based functions due to a lack of available compiler licenses, with error:

Error: A license for CCompL is not available now (-15,570,115)

This makes batch runs virtually impossible to implement in this way. In any case, I have also been asked to avoid recompiling at each run. Is this possible to set up? The necessary files appear to remain in the .pyxbld directory in this qutip version, but RHS_filename option does not appear to be used with mcsolve.

[Paul Nation]

Doing the time-dependent compiling on a cluster was not considered when we originally wrote the code.  As it stands right now, you will probably need to compile once per cluster node that you are running on, and then can use “rhs_reuse=True” in the Options class to the mcsolver that can avoid recompiling provided that only the args change.

This does however, expose some holes that can be easily cleaned up.  For example, currently it is possible to generate a RHS function using the rhs_generate function, but the mcsolve version was removed when we rewrote the code and was forgotten about.  In principle, one could generate the RHS function once and then pass this to the cluster nodes.  However, this will require some code revision.

We never thought about compilation being an issue as it is fast, and we always assumed using the GCC compiler.  It is unfortunate that your institute uses the Intel compiler with a limited license.  But at the same time, it does point toward areas that need fixing.

Paul

--
You received this message because you are subscribed to the Google Groups "QuTiP: Quantum Toolbox in Python" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qutip+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paul Nation]

I should probably also mention that it is possible to run the mcsolver distributively using the parallel_map method that uses the IPython ipcluster engine.  This probably does not work in your case though, and the documentation on how to do this is yet to be written (scheduled for v 3.3).

Paul

 

On Apr 1, 2015, at 6:47 AM, Paul Haljan <paul....@gmail.com> wrote:

[Paul Haljan]

Thanks for the quick replies. I wasn't sure if there was indeed a different functionality for mesolve vs mcsolve in this regard, or if I was just missing something. The suggestion to use gcc compiler seems like the way to go to get around the license limitations, independent of the other issues.

Thanks

-p