Change in the published Linux binary builds with 2025.1?

[tibo...@gmail.com]

Dear Molpro developers,

In the past the Linux binary releases had both an MPI sockets and an MPI-PR build.

E.g.

molpro-mpp-2024.3.0.linux_x86_64_mpipr.sh.gz

and

molpro-mpp-2024.3.0.linux_x86_64_sockets.sh.gz

But it looks like now there is only one Linux binary, with no indication of the type of build:

molpro-mpp-2025.1.0.linux_x86_64.sh.gz

I could not find any note about changes in parallelism or binary release packaging in the release notes. So far we have been using the sockets version exclusively, so if the MPI-PR build is discontinued that is OK, but I am curious about this change in distribution.

Thanks,

Tibor Győri

[tibo...@gmail.com]

As a follow-up, it looks like there have been notable changes in parallelism that are not mentioned in the release notes.

For example the "default implementation of scratch files" has changed from "sf" to "df", and now one MPI process is always designated as a "helper process" by default.

[Peter Knowles]

Dear Tibor

Molpro is built using the Global Arrays package, and the sockets/MPI-PR flavours represented two of the different ways of configuring GA. As you have observed, MPI-PR requires helper process(es) to serve one-sided memory access.

We recently took the decision not to distribute sockets binaries any more. MPI-PR is the configuration recommended at https://globalarrays.readthedocs.io/en/latest/Section1-BuildInstructions.html#selecting-the-underlying-one-sided-runtime , and, as far as we understand, is the only one that is asserted to be supported.  https://www.molpro.net/manual/doku.php?id=running_molpro_on_parallel_computers explains additionally how configuring memory for Molpro can be slightly simpler when using MPI-PR.  We are aware of some failures on large memory-intensive jobs using runtimes other than MPI-PR, and feel that the correct thing to do is to make available only binaries that use MPI-PR.

The helper processes of course consume processor cycles. On modern hardware, and especially with the typical high data-movement characteristics of most quantum chemistry algorithms, it should usually be the case that processing cycles are not the resource type that limits execution speed.

Peter

On 4 Mar 2025, at 14:23, tibo...@gmail.com <tibo...@gmail.com> wrote:

External email to Cardiff University - Take care when replying/opening attachments or links.

Nid ebost mewnol o Brifysgol Caerdydd yw hwn - Cymerwch ofal wrth ateb/agor atodiadau neu ddolenni.

-- 
You received this message because you are subscribed to the Google Groups "molpro-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to molpro-user...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/molpro-user/b05456c4-178c-4cd0-9d45-9119f838fcb2n%40googlegroups.com.

[tibo...@gmail.com]

Dear Peter,

Thank you for explaining the rationale behind this change. I think this would be worth adding to the "Recent changes" page of the docs, for the sake of users who might not be monitoring this forum.
I might have further questions about the parallelism implementation in the future. I still need to reread the docs, but the interactions between different MPI/GA/disk vs. non-disk(?) implementations for all the different subprograms of Molpro feel like a bit of a web to me right now.

> The helper processes of course consume processor cycles. On modern hardware, and especially with the typical high data-movement characteristics of most quantum chemistry algorithms, it should usually be the case that processing cycles are not the resource type that limits execution speed.

I am curious, do you mean the data movement between MPI processes? So far, we have only used parallelism within a single host, so no networking required. Data movement ought to be extremely fast within a single host, unless something is misconfigured or buggy. So I am surprised that would be the rate-limiting step.
Or do you mean RAM bandwidth (or disk bandwidth) under data movement?

Thanks,
Tibor

[Peter Knowles]

On 6 Mar 2025, at 13:38, tibo...@gmail.com <tibo...@gmail.com> wrote:

Dear Peter,

Thank you for explaining the rationale behind this change. I think this would be worth adding to the "Recent changes" page of the docs, for the sake of users who might not be monitoring this forum.

This will be done shortly.

I might have further questions about the parallelism implementation in the future. I still need to reread the docs, but the interactions between different MPI/GA/disk vs. non-disk(?) implementations for all the different subprograms of Molpro feel like a bit of a web to me right now.

> The helper processes of course consume processor cycles. On modern hardware, and especially with the typical high data-movement characteristics of most quantum chemistry algorithms, it should usually be the case that processing cycles are not the resource type that limits execution speed.

I am curious, do you mean the data movement between MPI processes? So far, we have only used parallelism within a single host, so no networking required. Data movement ought to be extremely fast within a single host, unless something is misconfigured or buggy. So I am surprised that would be the rate-limiting step.
Or do you mean RAM bandwidth (or disk bandwidth) under data movement?

I meant simply in-node memory bandwidth, together with the fact that the number of cores on a node is now typically quite large.

Peter

Thanks,
Tibor