Re: [nmag-users] Hysteresis loops stalling and MPI convergence

[Fangohr H.]

Dear Graham,

thank you for the detailed report.

Some thoughts that may or may not help.

I agree that the mesh looks of reasonable quality based on the nmeshpp -i -q data you have sent.

To compile the 0.2.1 tar ball on recent Ubuntu you need to change one file (see https://groups.google.com/forum/#!topic/nmag-users/X1m5b2kmEEU), but otherwise it should just work. This should address the question whether the problem originates from the newer versions of support libraries you have used.

Note that Nmag’s parallel performance gain is not very good: you could try (if you have 8 nodes/cores) whether running 2 simulations on 4 nodes is more efficient than running 1 over all 8. If you have the memory and enough parameter space to explore, then 8 different runs on 8 nodes may be most efficient.

The two situations in which we have seen convergence issues in the past are (i) if energy is pumped into the system (for example through spin-torque-coupling terms) and (ii) occasionally if the tolerances for the demag calculation and the time integration are inconsistent: imagine that the time integrator is being told to achieve a certain accuracy with the parameters ts_rel_tol and ts_abs_tol (see http://nmag.soton.ac.uk/nmag/0.2/manual/html/example_tolerances/doc.html#example-tolerances). Suppose the absolute requested error is 10^-9. If at the same time, the required accuracy for the demag calculation is only 10^-6, then the demagnetisation field will effectively be random in the precision digits smaller than 10^-6 (i.e. for 10^-7, 10^-8, etc). The time integrator then has a difficult time to carry out the time integration as the right hand side of the ODE (that is essentially the LLG equation in which H_Demag enters via the effective field) appears to make random changes. In this situation, one may run into convergence issues.

The manual briefly comments on setting the tolerances for the demag calculation (see ksp_tolerances in http://nmag.soton.ac.uk/nmag/0.2/manual/html/command_reference.html#simulation). I try to attach a document that gives a full example of setting the demag tolerances. Setting these sensibly is not easy, but at least you should have all the information that is required with this email.

Maybe this helps.

Thanks,

Hans






On 20 Aug 2014, at 04:54, Graham Rowlands <grah...@gmail.com> wrote:

> Greetings nmag users,
>
> I've been observing a few strange behaviors lately and wanted to see if anyone had any insight. I have investigated the second problem more than the first at this point.
>
> 1.) Convergence for simple meshes (1k-8k nodes) sometimes fails with larger number of processors (~8) but appears consistently fine with fewer (~4).
>
> 2.) Larger meshes (50k-100k nodes) get stuck in hysteresis loops. Occasionally I see errors from CVODE about convergence, but normally this occurs without any useful information being printed on INFO, INFO2, or DEBUG channels. The simulation may work fine for several field steps (with convergence=True), but then at some unpredictable step it prints
>
> INFO hysteresis: starting new stage: field = [xx xx xx]
>
> and sits there spinning its wheels indefinitely without any indication of trouble in the various log files. I'm doing the calculations with a large magnetic damping and have disabled the precessional term in the LLG eqn.
>
> The meshes that tend to fail are usually somewhat complicated in shape, but I've spent a good amount of time attempting to ensure that they are of high quality. A zip file containing both vtk and nmesh.h5 formatted versions of an example mesh can be found at https://dl.dropboxusercontent.com/u/10789691/problem-mesh.zip. The features of the mesh are somewhat small compared to the exchange length, so perhaps there are some stiffness problems. Please note that I've seen problems with simpler meshes, but not for a completely flat mesh. Here is the output from nmeshpp -i -q from the files linked above:
>
> ====== Info output: =====================================
> 3-dimensional mesh
> 43582 volume elements (3d)
> 12044 surface elements (2d)
> 10544 points
> 1 simplex regions ([1])
> 2 point regions ([-1, 1])
> 2 region volumes ([0.0, 104.04922935933718])
> 6024 boundary points (-> BEM size<= 277MB)
> 0 periodic points (mirage=0, total=0)
> a0: average=0.260762, std=0.093217, min=0.085937, max=0.774413
> ====== Quality output: ===================================
> [qual interval] counts = probability
> [ 0.000- 0.100] 0 = 0.00%
> [ 0.100- 0.200] 0 = 0.00%
> [ 0.200- 0.300] 1 = 0.00% *
> [ 0.300- 0.400] 8 = 0.02% *
> [ 0.400- 0.500] 103 = 0.24% *
> [ 0.500- 0.600] 642 = 1.47% **
> [ 0.600- 0.700] 3137 = 7.20% *******
> [ 0.700- 0.800] 9923 =22.77% **********************
> [ 0.800- 0.900] 18781 =43.09% ****************************************
> [ 0.900- 1.000] 10987 =25.21% ************************
>
> I'm curious if the problems are somehow related to my own installation, which is on Ubuntu 14.04. I had to update a few of the packages in order to successfully compile nmag, i.e.
> PYTHON_PKG=pkgs/Python-2.7.7.tar.gz
> IPYTHON_PKG=pkgs/ipython-2.1.0.tar.gz
> PYTHON_CYTHON_PKG=pkgs/Cython-0.20.1.tar.gz
> PYTHON_NUMEXPR_PKG=pkgs/numexpr-1.4.1.tar.gz
> PYTHON_PYTABLES_PKG=pkgs/tables-2.4.0.tar.gz
> PYTHON_NUMPY_PKG=pkgs/numpy-1.8.1.tar.gz
> PYTHON_SCIPY_PKG=pkgs/scipy-0.14.0.tar.gz
> PYTHON_PY_PKG=pkgs/py-0.9.1.tar.gz
> PYTHON_PLY_PKG=pkgs/ply-3.3.tar.gz
> SUNDIALS_PKG=pkgs/sundials-2.3.0.tar.gz
> HDF5_PKG=pkgs/hdf5-1.8.13.tar.bz2
>
> I suppose that SUNDIALS is implicated with the CVODE convergence issues. I believe choosing that particular version was necessary to maintain API compatibility. Soon I will be trying the nmag virtual machine image to see if it is indeed my own environment.
>
> Other things I've tried that didn't work:
>
> 1.) Running without MPI
> 2.) Obsessing over the mesh quality
> 3.) Reordering the nodes with nmeshpp --reordernodes
> 4.) Decreasing tolerances
>
> So, it anyone has any good ideas for debugging steps I would be very grateful! I'm new to finite element calculations, so it's very possible that I've made some glaring errors.
>
> Best and Thanks,
> Graham
>
> ---
> Graham Rowlands
> Postdoctoral Research Associate
> Buhrman Lab, Applied & Engineering Physics
> Cornell University
Ithaca NY, USA