Quantum Trajectories, Monte Carlo solver and memory issues

[Alberto Biella]

Dear all,

I am writing you for some suggestions about the Monte Carlo solver mcsolve.

I am using it a lot to run quantum trajectories for a "large" 1D system of fermions as you can see from our recent preprint https://arxiv.org/abs/2104.07929 .

mcsolve is very effective for our problem and allowed us to push to L=10 sites (dimension of the Hilbert space being 4^10) for large times. 

Despite we do not ask to save nor the trajectories wavefunctions nor the final states (we just want the averaged observables), we get a huge memory usage that cause the failure of the execution after some % of the procedure, on many cluster we are trying to run the code. 

Given that we do not save the wavefunctions, I do not see what is using this memory. I see that the solver saves the times at which the jumps take place for each trajectory but I do not think this is the problem.

I would like to know if there is some options I am not aware about to limit this memory usage and what is its origin.

Tanks in advance to who can help me !

Alberto

[Simon Cross]

Hi Alberto,

Do you have a short code example that shows the issue that you can
share? That would help us either explain what is happening or fix the
issue.

Yours sincerely,
Simon Cross

[Alberto Biella]

Dear Simon,

thank you for your interest.

Here attached you will find a clean version of the code that fail when mcsolve is called.

Also attached the typical output file for L=10.

Have a nice day,

Alberto

[Eric]

Hi Alberto,

There is 2 issues with mcsolve that make it not memory efficient.

1- It always store the final state for each trajectories during the evolution and only check if you want them after.

2- Even if you only want the average of all trajectories, it store the results of each trajectories and compute the averages after they are all computed.

(1) Should be easy to fix, I opened an issue so you can track progress.

But (2) will probably have to wait until the next major version, it require significant change in the way qutip runs trajectories in parallel.

Eric

[Alberto Biella]

Dear Eric,

thanks a lot for the work.

I see from https://github.com/qutip/qutip/issues/1667 that you made progress on the issues (1) and (2) and you eventually fixed them. I understand that you will make this available in the next version of Qutip 4.7.

However, I was wondering if we could manage to test the new version of the mcsolve.pyx you created. In order to do so, I downloaded it and tried to cythonize it in order to get the *.cpp and *.so file (in the cy/ folder) needed to make it work (as far as I understand). We failed :(

I was wondering if you can suggest a way to test the mcsolve routine before the release of Qutip 4.7.

Thanks again for your work,

have a nice day,

Alberto

PS. I put in cc my collaborators on this project, Lorenzo and Leonardo

 

--
You received this message because you are subscribed to a topic in the Google Groups "QuTiP: Quantum Toolbox in Python" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qutip/tzU_6BcbUrs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qutip+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qutip/fededf28-a210-41ba-b7d2-774cbbc3c559n%40googlegroups.com.

--

AB