[llvm-dev] Getting a combined llvm-cov report for a set of tests

[Chris Brown via llvm-dev]

I am working on a large software project with many thousands of tests. A common workflow using gcov as a coverage tool is:

1) Build code with coverage instrumentation

2) Run *all* the tests. Gcov aggregates the line coverage counters together into its .gcda files

3) Generate a report for some set of files using those combined gcda files.

4) Analyze the report to find files/sections of code that are not covered by any test.

 

Using llvm's source-based code coverage, I don't believe I can accomplish steps 3-4. Each binary produces a .profraw file which contains profile counts, but the mapping of those profile counts to files is contained inside the binary.  I can use llvm-profdata merge to merge multiple .profraw files together, but the report can still only be generated for a single binary, and so only the files contained in that binary's __llvm_covmap section will be produced as part of the report.

Am I correct in this thinking? Is there a way to get a single coverage report for a set of multiple binaries? I discovered the undocumented 'llvm-cov convert-for-testing' command which generates 'covmapping files', which appear to be the contents of __llvm_covmap with an additional header. Maybe it's possible to merge multiple of these covmapping files and generate a report from that data (plus the merged profraw files)? Or perhaps I could use 'llvm-cov export' on each binary individually and manually merge the results somehow (although in that case, I would not be able to 'import' the merge to create the usual report) ? Are there other suggestions, or is this a planned feature in 4.0? I found some discussion on this thread, but I am not sure if this is asking the same thing:

https://groups.google.com/forum/#!topic/llvm-dev/r4qOdN-y9hc

Thanks!

[Vedant Kumar via llvm-dev]

Hi,

There isn't yet a way to generate a single coverage report for multiple
binaries. We're planning on adding this functionality to llvm-cov soon. The
patch is up for review here: https://reviews.llvm.org/D25086.

> Maybe it's possible to merge multiple of these covmapping files and generate a report from that data (plus the merged profraw files)?

It also isn't yet possible to merge multiple 'covmapping' files. Do you have
other use-cases for this functionality (apart from using it to generate a
single report given multiple binaries)?

> I found some discussion on this thread, but I am not sure if this is asking the same thing:
> https://groups.google.com/forum/#!topic/llvm-dev/r4qOdN-y9hc

Yes, that thread is about the same topic. The person who started it is the
author of the patch I linked to :).

best,
vedant

> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev