Generating coverage report for Cython files using a meson-python build backend

[Joseph Capriotti]

I've been trying, and struggling to generate accurate coverage reports for our project's Cython files. It only seems to generate coverage reports for the lines that have interaction with python code. (not any lines which are translated to pure C). I do not know what else is happening or where this process is wrong.

The project uses meson-python as the build backend. One of the intended designs of the meson system is that it doesn't put built files in the source directories (meaning the transpiled C/C++ codes end up in a build directory). meson-python supports extension modules with in place building by installing hooks to the compiled codes that live in the build directory, (but by default these do not exist next to the transpiled code either). From what I understand the Cython.Coverage tool looks for the transpiled codes to live next to their source pyx files, therefore I've added a step that cythonizes (with '--directive linetrace=true') those sources again so that transpiled code ends up next to the pyx source. This process seems to allow coverage.py tool to find the correct c/cpp file.

During a coverage build, I tell meson to add the `--directive linetrace=true` directive to the cython project arguments, and `-DCYTHON_TRACE_NOGIL=1` to both the C and C++ project arguments. The commands that the ninja build executes "look" correct, for example. I've tried both `debug` and `release` meson builds, and neither makes a difference in the coverage.

For reference here are my attempts at getting it working:  https://github.com/simpeg/discretize/pull/357. 

This is the relevant azure-pipeline building commands:https://dev.azure.com/simpeg/discretize/_build/results?buildId=4196&view=logs&j=34714459-c2a6-52d6-dff9-d2b4dab960d2&t=8e9d1786-6735-55db-3442-b3b9b55a3aba&l=11

And the coverage reported for the cython extensions.

https://app.codecov.io/gh/simpeg/discretize/pull/357?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=simpeg

As you can see it completely misses coverage of `simplex_helpers.pyx`, and the other files only report coverage on lines that had any (even the smallest) interaction with python.

[Oscar Benjamin]

On Sat, 25 May 2024 at 20:20, Joseph Capriotti
<josephrc...@gmail.com> wrote:
>
> I've been trying, and struggling to generate accurate coverage reports for our project's Cython files. It only seems to generate coverage reports for the lines that have interaction with python code. (not any lines which are translated to pure C). I do not know what else is happening or where this process is wrong.
>
> The project uses meson-python as the build backend.

See https://github.com/cython/cython/issues/6186

One problem is that in an out of tree build with src layout Cython
only outputs the basename of the files.

--
Oscar