Questions about running simulation and depletion in parrellel using python script.

[Yue JIN]

Hi all,

I had run OpenMC using 2 computers, a local computer with AMD Ryzen 9 3900X (12 cores, 24 threads) and a remote virtual machine with Intel(R) Xeon(R) Silver 4114 (16 cores, 16 threads). The implemention of MPI is OpenMPI. It took 50 seconds for a simple bundle assembly.

The command line I used is “mpirun -use-hwthread-cpus --map-by slot:PE=8 --hostfile /home/yuejin/Cloud/mpi_config_file openmc”.

In the host file, number of slots of local node and remote node are 3 and 2 respectively.

If I run OpenMC on local computer alone with 1 MPI process * 24 OpenMP threads, total time elapsed is 80 s. So the timings are resonable.

 

I’m new to MPI. I wonder if there is a better way to write my command line? Is there anything that I missed in arguments which can affect the performance? Maybe there is a way to set (1 MPI * 24 threads + 1 MPI * 16 threads) instead of (3+2)MPI*8 threads that I set? Which one is better?

 

Secondly, I didn’t find how to add MPI arguments in openmc.depletion.Integrator. Can anyone show me an example how to run depletion in parrellel using two computers?

Regards,

Yue.

[Yue JIN]

I tried "mpirun -use-hwthread-cpus --map-by slot:PE=8 --hostfile /home/yuejin/Cloud/mpi_config_file python3 build.py". But it raised an error: 'No material exists with ID=5.'

Here are the python script and output file.

[Yue JIN]

I found that even using one computer can cause the same problem. The more cores I use, the greater the probability of an error. For example, if  I try  "mpirun -np 12 python3 build.py" 10 times, it may raise error 8 or 9 times. If I use  "mpirun -np 10 python3 build.py"  10 times, it may raise this error 4 or 5 times.

[Yue JIN]

Finally, I solved this issue by removing all the "export_to_xml" in the python script. Xml exported by a process will overwrite the modified depletion material xml generated by another process.

However, I still have some concerns. 

1.  In the following output during initialization:

 --- Reading c_H_in_H2O from

 --- /home/yuejin/Cloud/nuclear_data_libraries/endfb71_hdf5/c_H_in_H2O.h5

 --- Minimum neutron data temperature: 900.000000 K

 --- Maximum neutron data temperature: 600.000000 K

Why Minimum neutron data temperature is higher than the maximum?

2. Some warnings appear in output of each statepoint

/usr/local/lib/python3.8/dist-packages/openmc/mixin.py:68: IDWarning: Another Filter instance already exists with id=2.

  warn(msg, IDWarning)

/usr/local/lib/python3.8/dist-packages/openmc/mixin.py:68: IDWarning: Another Filter instance already exists with id=6.

  warn(msg, IDWarning)

/usr/local/lib/python3.8/dist-packages/openmc/mixin.py:68: IDWarning: Another Filter instance already exists with id=3.

  warn(msg, IDWarning)

Is it because I set too many crossection tallies? Will these confrontations affect the final results?