PFLOTRAN likes an odd number of cores (?)

[James Hanson]

Afternoon all,

I am going to commit the ultimate cardinal sin by making a post without including my input script as it is, regrettably, proprietary knowledge of the company I work for... 

Anyways, I am well aware that PFLOTRAN does not scale for problems with few DOFs,  however I have also found that, for smaller problems at least, that PFLOTRAN prefers an odd number of cores. I have seen a similar effect in other codes, but for even numbers of cores... Any insight what is happening here? 

Attached is a bar chart I made of the simulated time achieved after the first 100 steps of a simulation run at various numbers of cores. What can be seen is that for even numbers of cores, the timesteps never grow significantly to span much simulated time (indeed, inspecting the .out, the Dt remains suppressed near the initial timestep, which in this case is 1.d-15 days)... Curious! As mentioned previously, one can also see the progressively worse performance with more cores...

I have been able to replicate the same effect on another machine (although this is unsurprising as both machines have the same version of Ubuntu and PFLOTRAN was installed using the same protocol)

The command I use to run is as follows: <path_to_mpirun> - np <number_of_cores> --bind-to core taskset -c 0-<number_of_cores> ...etc...

I wish everyone here all the best in their PFLOTRAN endeavours...

James