PFLOTRAN likes an odd number of cores (?)

[James Hanson]

Afternoon all,

I am going to commit the ultimate cardinal sin by making a post without including my input script as it is, regrettably, proprietary knowledge of the company I work for... 

Anyways, I am well aware that PFLOTRAN does not scale for problems with few DOFs,  however I have also found that, for smaller problems at least, that PFLOTRAN prefers an odd number of cores. I have seen a similar effect in other codes, but for even numbers of cores... Any insight what is happening here? 

Attached is a bar chart I made of the simulated time achieved after the first 100 steps of a simulation run at various numbers of cores. What can be seen is that for even numbers of cores, the timesteps never grow significantly to span much simulated time (indeed, inspecting the .out, the Dt remains suppressed near the initial timestep, which in this case is 1.d-15 days)... Curious! As mentioned previously, one can also see the progressively worse performance with more cores...

I have been able to replicate the same effect on another machine (although this is unsurprising as both machines have the same version of Ubuntu and PFLOTRAN was installed using the same protocol)

The command I use to run is as follows: <path_to_mpirun> - np <number_of_cores> --bind-to core taskset -c 0-<number_of_cores> ...etc...

I wish everyone here all the best in their PFLOTRAN endeavours...

James

[James Hanson]

Just realised the image may not render correctly in the original message... please see attached copy

[Hammond, Glenn E]

James,

It is challenging to consider what might be causing the performance below without some additional detail regarding problem size, platform and operating system, and the process models employed in the problem scenario…no need to divulge any proprietary info.

Also, can you replicate this performance on any of the short course exercises?

Glenn

From: pflotra...@googlegroups.com <pflotra...@googlegroups.com> on behalf of James Hanson <jhans...@gmail.com>
Date: Wednesday, June 17, 2026 at 1:26 AM
To: pflotran-users <pflotra...@googlegroups.com>
Subject: [pflotran-users: 8800] Re: PFLOTRAN likes an odd number of cores (?)

Check twice before you click! This email originated from outside PNNL.

--
You received this message because you are subscribed to the Google Groups "pflotran-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pflotran-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pflotran-users/f7445506-c62c-467a-a6fb-a52f405fe7abn%40googlegroups.com.

[James Hanson]

Hi Glenn,

I gave it a go on the co2-mineralization shortcourse at pflotran/shortcourse/exercises/co2/sco2/co2-mineralization. I chose this shortcourse as it is somewhat representative of the problem I am solving in my own work. However, I did not find the same pattern, the timesteps used were virtually identical no matter the number or parity of cores used. However, the discretisation for the short course was 20x1x25 (=500 cells), whereas my own work has a total of 80,000 cells, so perhaps this creates some difference? 

Kind regards,

James

[Hammond, Glenn E]

The difference between 500 and 80,000 cells is certainly significant. However, I would typically expect poorer performance with the smaller cell count, as larger workloads generally enhance strong scaling. Would you be able to create an input deck that reproduces this result?

Additionally, could you specify the computing platform and operating system you are using? Are you running this on a workstation with Ubuntu?

Glenn

From: pflotra...@googlegroups.com <pflotra...@googlegroups.com> on behalf of James Hanson <jhans...@gmail.com>

Date: Tuesday, June 23, 2026 at 3:28 AM
To: pflotran-users <pflotra...@googlegroups.com>
Subject: Re: [pflotran-users: 8808] Re: PFLOTRAN likes an odd number of cores (?)

To view this discussion visit https://groups.google.com/d/msgid/pflotran-users/123af2f0-4073-46ee-90f0-811d19293207n%40googlegroups.com.

[James Hanson]

Hi Glenn,

Might give this a try. I am indeed using Ubuntu :)

James

You received this message because you are subscribed to a topic in the Google Groups "pflotran-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pflotran-users/wQsdcFZrRSI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pflotran-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pflotran-users/BY3PR09MB7555A728FBD88E9AA52B6FD39AEC2%40BY3PR09MB7555.namprd09.prod.outlook.com.

[Hammond, Glenn E]

James,

Would you be willing to send me the input deck that produces this issue directly, rather than through the mailing list? Please feel free to generalize or modify any names as needed to avoid sharing sensitive information.

Glenn

From: pflotra...@googlegroups.com <pflotra...@googlegroups.com> on behalf of James Hanson <jhans...@gmail.com>

Date: Thursday, June 25, 2026 at 9:23 AM
To: pflotra...@googlegroups.com <pflotra...@googlegroups.com>
Subject: Re: [pflotran-users: 8816] Re: PFLOTRAN likes an odd number of cores (?)

To view this discussion visit https://groups.google.com/d/msgid/pflotran-users/CABs6btVLFRbP0m2RA9sxde7bRXHBn7pHuKjKL-bhFZ14nPUZEg%40mail.gmail.com.