Pipe test case crashing with ioshape error

[Robert Sawko]

Dear PyFR developers,

I was wondering if you could help me with my test case. I have a modest OLCF
Summit allocation to test strong scaling of ACM for an internal flow simulation
and I am trying to set it up in PyFR using Loppi et al files.

My target geometry is a double elbow, but I am starting with a pipe. I am
attaching my gmsh and PyFR ini. I use

gmsh -o pipe.msh -f msh22 -3 pipe.geo

to mesh and then import and partition into 2 subdomains using PyFR. Running
with CUDA backend though gives me the following error

File "/ccs/proj/cfd141/apps/pyfr/1.10.0/pyfr/pyfr", line 112, in main
args.process(args)
File "/ccs/proj/cfd141/apps/pyfr/1.10.0/pyfr/pyfr", line 245, in process_run
args, NativeReader(args.mesh), None, Inifile.load(args.cfg)
...
File "/ccs/proj/cfd141/apps/pyfr/1.10.0/pyfr/solvers/base/system.py", line 193, in _gen_kernels
kernels[pn, kn].append(kgetter())
File "/ccs/proj/cfd141/apps/pyfr/1.10.0/pyfr/solvers/baseadvec/elements.py", line 61, in <lambda>
out=slicem(self._scal_fpts, s)
File "/ccs/proj/cfd141/apps/pyfr/1.10.0/pyfr/solvers/base/elements.py", line 131, in _slice_mat
if len(mat.ioshape) >= 3:
AttributeError: 'CUDAMatrixSlice' object has no attribute 'ioshape'

I am attaching my geo and ini file. Please let me know if anything strikes you.
My guess is that I am still messing something up with the mesh.

Best wishes,
Robert
--
Dr Robert Manson-Sawko
Research Staff Member,
IBM Research Europe
Daresbury Laboratory
Keckwick Lane, Warrington
WA4 4AD
United Kingdom
--
Email (IBM): RSa...@uk.ibm.com
Email (STFC): robert...@stfc.ac.uk
Phone (office): +44 (0) 1925 60 3967
Phone (mobile): +44 759 301 0452
Profile page:
http://researcher.watson.ibm.com/researcher/view.php?person=uk-RSawko
--Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Freddie Witherden]

Hi Robert,

Can you try removing the div-flux option from the anti-alias directive?
This is a piece of functionality we've removed in the most recent
version of PyFR.

Regards, Freddie.

[Robert Sawko]

Thanks Freddie. Yes. That was precisely the problem. It's running now and I
should have mentioned that I am trying 1.10 version.

I will play with other options now.

Best wishes,
Robert
--
Dr Robert Manson-Sawko
Research Staff Member,
IBM Research Europe
Daresbury Laboratory
Keckwick Lane, Warrington
WA4 4AD
United Kingdom
--
Email (IBM): RSa...@uk.ibm.com
Email (STFC): robert...@stfc.ac.uk
Phone (office): +44 (0) 1925 60 3967
Phone (mobile): +44 759 301 0452
Profile page:
http://researcher.watson.ibm.com/researcher/view.php?person=uk-RSawko
--

-----pyfrmai...@googlegroups.com wrote: -----
To: pyfrmai...@googlegroups.com
From: Freddie Witherden
Sent by: pyfrmai...@googlegroups.com
Date: 09/30/2020 11:17PM
Subject: [EXTERNAL] Re: [pyfrmailinglist] Pipe test case crashing with ioshape error

--
You received this message because you are subscribed to the Google Groups "PyFR Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyfrmailingli...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pyfrmailinglist/51014227-a4b6-e24a-de44-fb87f793e150%40witherden.org .

[Robert Sawko]

Freddie,

Unfortunately, I seem to be producing lots of NaNs almost immediately. Is there
anything else in this Loppi setting that needs to be updated?

Also, I've used progressive mesh refinement for near wall region. Is that
something that could lead to NaNs? Should I go back to more uniform cell size?

[Robert Sawko]

PS. Scratch that. After playing with the parameters I seem to be producing some
correct behaviour for the pipe. I will run a bit more thorough tests and
report.