inc_cylinder_2d testcase fails:

[Yuri]

On FreeBSD 11.2 (with the clang compiler) this command fails:







$ pyfr run -b openmp -p inc_cylinder_2d.pyfrm inc_cylinder_2d.ini

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pyfr/util.py", line 33, in __call__
    res = cache[key]
KeyError: (<function OpenMPKernelProvider._build_kernel at 0x80d54d158>, b'\x80\x03X\t\x00\x00\x00gimmik_mmq\x00X\xaf\x07\x00\x00\nvoid\ngimmik_mm(int ncol,\n         const double* restrict b, int ldb,\n         double* restrict c, int ldc)\n{\n    double dotp;\n\n    #pragma omp parallel for simd private(dotp)\n    for (int i = 0; i < ncol; i++)\n    {\n        dotp = 1.478830557701236*b[i + 0*ldb] + -0.6666666666666666*b[i + 3*ldb] + 0.1878361089654305*b[i + 6*ldb];\n        c[i + 0*ldc] = dotp;\n        dotp = 1.478830557701236*b[i + 1*ldb] + -0.6666666666666666*b[i + 4*ldb] + 0.1878361089654305*b[i + 7*ldb];\n        c[i + 1*ldc] = dotp;\n        dotp = 1.478830557701236*b[i + 2*ldb] + -0.6666666666666666*b[i + 5*ldb] + 0.1878361089654305*b[i + 8*ldb];\n        c[i + 2*ldc] = dotp;\n        dotp = 0.1878361089654305*b[i + 0*ldb] + -0.6666666666666666*b[i + 1*ldb] + 1.478830557701236*b[i + 2*ldb];\n        c[i + 3*ldc] = dotp;\n        dotp = 0.1878361089654305*b[i + 3*ldb] + -0.6666666666666666*b[i + 4*ldb] + 1.478830557701236*b[i + 5*ldb];\n        c[i + 4*ldc] = dotp;\n        dotp = 0.1878361089654305*b[i + 6*ldb] + -0.6666666666666666*b[i + 7*ldb] + 1.478830557701236*b[i + 8*ldb];\n        c[i + 5*ldc] = dotp;\n        dotp = 0.1878361089654305*b[i + 0*ldb] + -0.6666666666666666*b[i + 3*ldb] + 1.478830557701236*b[i + 6*ldb];\n        c[i + 6*ldc] = dotp;\n        dotp = 0.1878361089654305*b[i + 1*ldb] + -0.6666666666666666*b[i + 4*ldb] + 1.478830557701236*b[i + 7*ldb];\n        c[i + 7*ldc] = dotp;\n        dotp = 0.1878361089654305*b[i + 2*ldb] + -0.6666666666666666*b[i + 5*ldb] + 1.478830557701236*b[i + 8*ldb];\n        c[i + 8*ldc] = dotp;\n        dotp = 1.478830557701236*b[i + 0*ldb] + -0.6666666666666666*b[i + 1*ldb] + 0.1878361089654305*b[i + 2*ldb];\n        c[i + 9*ldc] = dotp;\n        dotp = 1.478830557701236*b[i + 3*ldb] + -0.6666666666666666*b[i + 4*ldb] + 0.1878361089654305*b[i + 5*ldb];\n        c[i + 10*ldc] = dotp;\n        dotp = 1.478830557701236*b[i + 6*ldb] + -0.6666666666666666*b[i + 7*ldb] + 0.1878361089654305*b[i + 8*ldb];\n        c[i + 11*ldc] = dotp;\n    }\n}\nq\x01]q\x02(cnumpy\nint32\nq\x03cnumpy\nint64\nq\x04h\x03h\x04h\x03e\x87q\x05.', b'\x80\x03}q\x00.')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pyfr", line 11, in <module>
    load_entry_point('pyfr==1.7.6', 'console_scripts', 'pyfr')()
  File "/usr/local/lib/python3.6/site-packages/pyfr/__main__.py", line 110, in main
    args.process(args)
  File "/usr/local/lib/python3.6/site-packages/pyfr/__main__.py", line 235, in process_run
    args, NativeReader(args.mesh), None, Inifile.load(args.cfg)
  File "/usr/local/lib/python3.6/site-packages/pyfr/__main__.py", line 216, in _process_common
    solver = get_solver(backend, rallocs, mesh, soln, cfg)
  File "/usr/local/lib/python3.6/site-packages/pyfr/solvers/__init__.py", line 16, in get_solver
    return get_integrator(backend, systemcls, rallocs, mesh, initsoln, cfg)
  File "/usr/local/lib/python3.6/site-packages/pyfr/integrators/__init__.py", line 46, in get_integrator
    return integrator(backend, systemcls, rallocs, mesh, initsoln, cfg)
  File "/usr/local/lib/python3.6/site-packages/pyfr/integrators/dual/multip.py", line 54, in __init__
    super().__init__(backend, systemcls, rallocs, mesh, initsoln, cfg)
  File "/usr/local/lib/python3.6/site-packages/pyfr/integrators/dual/controllers.py", line 48, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pyfr/integrators/dual/controllers.py", line 15, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pyfr/integrators/dual/steppers.py", line 8, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pyfr/integrators/dual/base.py", line 10, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pyfr/integrators/base.py", line 58, in __init__
    self._init_system(systemcls, backend, rallocs, mesh, initsoln)
  File "/usr/local/lib/python3.6/site-packages/pyfr/integrators/dual/multip.py", line 208, in _init_system
    *args, nreg=self.nreg + (2 if l != self._order else 0), cfg=cfg
  File "/usr/local/lib/python3.6/site-packages/pyfr/solvers/base/system.py", line 65, in __init__
    self._gen_kernels(eles, int_inters, mpi_inters, bc_inters)
  File "/usr/local/lib/python3.6/site-packages/pyfr/solvers/base/system.py", line 174, in _gen_kernels
    kernels[pn, kn].append(kgetter())
  File "/usr/local/lib/python3.6/site-packages/pyfr/solvers/baseadvec/elements.py", line 57, in <lambda>
    out=self._scal_fpts
  File "/usr/local/lib/python3.6/site-packages/pyfr/backends/base/backend.py", line 166, in kernel
    return kern(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pyfr/backends/openmp/gimmik.py", line 34, in mul
    [np.int32] + [np.intp, np.int32]*2)
  File "/usr/local/lib/python3.6/site-packages/pyfr/util.py", line 35, in __call__
    res = cache[key] = self.func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pyfr/backends/openmp/provider.py", line 13, in _build_kernel
    mod = SourceModule(src, self.backend.cfg)
  File "/usr/local/lib/python3.6/site-packages/pyfr/backends/openmp/compiler.py", line 59, in __init__
    call_capture_output(self.cc_cmd(cname, lname), cwd=tmpdir)
  File "/usr/local/lib/python3.6/site-packages/pytools/prefork.py", line 223, in call_capture_output
    return forker.call_capture_output(cmdline, cwd, error_on_nonzero)
  File "/usr/local/lib/python3.6/site-packages/pytools/prefork.py", line 180, in call_capture_output
    error_on_nonzero)
  File "/usr/local/lib/python3.6/site-packages/pytools/prefork.py", line 162, in _remote_invoke
    raise result
pytools.prefork.ExecError: error invoking 'cc -shared -std=c99 -Ofast -march=native -fopenmp -fPIC -o libtmp.so tmp.c -lm': status 1 invoking 'cc -shared -std=c99 -Ofast -march=native -fopenmp -fPIC -o libtmp.so tmp.c -lm': b'warning: loop not vectorized: failed explicitly specified loop vectorization [-Wpass-failed=loop-vectorize]\n1 warning generated.\n/usr/bin/ld: cannot find -lomp\ncc: error: linker command failed with exit code 1 (use -v to see invocation)\n'


[Freddie Witherden]

Hi Yuri,

On 21/09/2018 06:11, Yuri wrote:
> pytools.prefork.ExecError: error invoking 'cc -shared -std=c99 -Ofast
> -march=native -fopenmp -fPIC -o libtmp.so tmp.c -lm': status 1 invoking
> 'cc -shared -std=c99 -Ofast -march=native -fopenmp -fPIC -o libtmp.so
> tmp.c -lm': b'warning: loop not vectorized: failed explicitly specified
> loop vectorization [-Wpass-failed=loop-vectorize]\n1 warning
> generated.\n/usr/bin/ld: cannot find -lomp\ncc: error: linker command
> failed with exit code 1 (use -v to see invocation)\n'

As the name suggests the OpenMP backend requires a compiler which
supports OpenMP. From the linker error above it appears as if libomp is
missing on your system (although this is an implementation detail). You
may need to either install an additional support library or a different
compiler version.

Regards, Freddie.

[Yuri]

Hi Freddie,

I do have libomp installed: /usr/local/lib/libomp.so.0

It looks like -L/usr/local/lib isn't passed to the linker.

LDFLAGS should be preserved from the time when the project was built, and passed to the linker every time thereafter.

Yuri

[Freddie Witherden]

Hi Yuri,

On 21/09/2018 09:47, Yuri wrote:
> I do have libomp installed: /usr/local/lib/libomp.so.0
> It looks like -L/usr/local/lib isn't passed to the linker.
> LDFLAGS should be preserved from the time when the project was built,
> and passed to the linker every time thereafter.

PyFR is not 'built' per se. All C code is generated, compiled, and
linked, at run time. Indeed, many users of PyFR just run it directly
from a clone of the git repository without every going through setup.py.

If your compiler requires additional flags you have a few options. One
is to simply export the relevant environmental variables before running
PyFR. Alternatively, in your .ini file you may set

[backend-openmp]
cflags = -L/usr/local/lib

as appropriate. (It says cflags however everything is done in one shot
and so linker flags are fine here, too.)

Regards, Freddie.

[Yuri]

I patched the FreeBSD port with -I/-L flags.

Thanks!