Strong scaling on Summit

[Robert Sawko]

Dear PyFR developers,

My colleague and I would like to setup two separate projects with PyFR:

1 external flow over a detailed car geometry,
2 internal incompressible flow through pipe bends with strong buoyancy.

I have asked about the latter before, although never managed to run appropriate
tests since then. We now have some preliminary tests on our medium size
internal P9+6xV100 cluster and we see perfect scaling. Currently, we are
evaluating whether our applications could benefit from Summit.

My question to you is whether any Summit results have been published or
presented in some format already? I would assume you have run benchmarks there.
We want to avoid work repetition if possible and are interested if there are
any lessons to be learnt for such installations.

Please let us know.

Best wishes,
Robert
--
Dr Robert Sawko
Research Staff Member, IBM
Daresbury Laboratory
Keckwick Lane, Warrington
WA4 4AD
United Kingdom
--
Email (IBM): RSa...@uk.ibm.com
Email (STFC): robert...@stfc.ac.uk
Phone (office): +44 (0) 1925 60 3967
Phone (mobile): +44 759 301 0452
Profile page:
http://researcher.watson.ibm.com/researcher/view.php?person=uk-RSawko
--Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Freddie Witherden]

Hi Robert,

On 09/03/2020 09:09, Robert Sawko wrote:
> My colleague and I would like to setup two separate projects with PyFR:
>
> 1 external flow over a detailed car geometry,
> 2 internal incompressible flow through pipe bends with strong buoyancy.
>
> I have asked about the latter before, although never managed to run appropriate
> tests since then. We now have some preliminary tests on our medium size
> internal P9+6xV100 cluster and we see perfect scaling. Currently, we are
> evaluating whether our applications could benefit from Summit.
>
> My question to you is whether any Summit results have been published or
> presented in some format already? I would assume you have run benchmarks there.
> We want to avoid work repetition if possible and are interested if there are
> any lessons to be learnt for such installations.

Please find attached a slide deck that we put together some months ago
with an initial scaling study. Note that since then it is my
understanding that the software stack on Summit has improved somewhat,
and we have also made improvements on the PyFR side.

Regards, Freddie.

[Robert Sawko]

Freddie,

Thanks for this. This is precisely what I was looking for. I've been also
testing CUDA Aware MPI but on simpler benchmarks, but internally rather than on
Summit.