Regarding code coverage on Windows

[YUN LI]

Devs,

I followed the design doc on Chromium.org to generate code coverage on Windows but failed. It seems the guide of generating code coverage is out of date. Could someone share with me whether our community takes care of code coverage on Windows? If yes, how can I generate it. If not, why don't  we take care of it and how can we make sure important cases are covered.

[Brett Wilson]

I don't believe there is code coverage hooked up. In the past, the
data generated from this has not been worth the effort of maintaining
the build configuration, which was very fragile. Also, most developers
don't work on Windows so Windows code coverage tools would be of
limited utility.

Brett

> --
> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev

[Sébastien Marchand]

It's possible to use SyzyCov to do this, Chris spent some time on this but there was too many platform specific files for the data to be meaningful (i.e. the coverage was low because of those files).

If you need/want coverage I can point you to some documentation (or examples) on how to use SyzyCov.

[YUN LI]

It would be great if you could share with me the SyzyCov way. I would like to give it a try.

[Sébastien Marchand]

You can look at https://github.com/google/syzygy/blob/master/syzygy/build/generate_coverage.py to see how we generate the coverage for our project (Syzygy), you should look at the '_CodeCoverageRunnerSyzygy' class in particular. 

You can grab the latest version of the Syzygy toolchain at https://syzygy-archive.commondatastorage.googleapis.com/index.html?path=builds/official/52a58e8bb19c5dcc97970bae4b8c00c5891f3568/

Here's an overview of the different steps:

1. Compile chrome with debugging symbols (i.e. set fastbuild=0 in your GYP_DEFINES)
2. instrument chrome.dll, chrome_child.dll and all the binaries you're interested in with instrument.exe (one of the Syzygy binaries). e.g.: instrument.exe --mode=COVERAGE --input-image=chrome.dll --output-image=${TEMP_DIR}/chrome.dll --no-augment-pdb"
3. Copy the instrumented image into the original directory.
4. Copy coverage_client.dll next to the instrumented images.
   
5. Start the call trace service: call_trace_service.exe spawn --trace-dir=${TEMP_DIR}/traces/ --instance-id=42
6. Run Chrome
7. call_trace_service.exe stop --instance-id=42
8. The coverage would be generated as LCOV files, you can use croc to generate a html version of it (look at the _GenerateHtml function in the script).

Sébastien Marchand | Software Developer | sebma...@google.com 

[Chris Hamilton]

A few more comments:

- If you specify --instance-id on the 'call_trace_service' command lines, you'll have to also specify an instance ID environment variable so the coverage client knows which service to connect to. This is done using the SYZYGY_RPC_INSTANCE_ID environment variable. See the script for details. If you specify no instance ID, then it tries to connect to the default 'blank' instance ID. This is usually fine, unless you want to separate where data goes for multiple concurrent processes.

- The coverage data is actually generated as .bin files in the --trace-dir specified to call_trace_service. You then need to process these using 'grinder.exe' in order to produce an LCOV coverage file. See the script for details.

- You probably want to instrument *all* of the .exe and .dll's in the output directory once you've built the *entirety* of the Chrome projects, which includes all unittests, etc. There may or may not be some issues with instrumenting or running some unittests, as it's been a while since we tried doing that (ie: almost 2 years ago).

Chris

[YUN LI]

Thanks to all. I'm working on that.

[Liuwei Lian]

Do you mean all platform Code Coverage reports are all down?

CodeCoverage - chromium - How to deal with test coverage figures.

https://code.google.com/p/chromium/wiki/CodeCoverage

all link are dead.

[zhaoqin]

You can try DrCov, a code coverage tool based on DynamRIO, a dynamic instrumentation platform.

So you do not have to recompile the code, so long there is debug information available, and it works on both Linux and Windows.

1. download dynamorio from http://www.dynamorio.org/

2. run you application with drcov tool

    .\bin32\drrun.exe -t drcov -- .\common.fib.exe

   and one drcov log file (e.g. drcov.common.fib.exe.73444.0000.proc.log) per process is generated.

3. run post-processing drcov2lcov on the logfile, something like

  D:\src\tmp\DynamoRIO-Windows-5.0.0-9>.\tools\bin32\drcov2lcov.exe -input drcov.common.fib.exe.73444.0000.proc.log -output coverage.info -src_filter "fib"

[DRCOV2LCOV] INFO(1):    Reading input files...

[DRCOV2LCOV] INFO(1):    Enumerating line info...

[DRCOV2LCOV] WARNING(1): Failed to enumerate lines for C:\Windows\syswow64\SspiCli.dll

[DRCOV2LCOV] INFO(1):    Writing output file...

  and you will get the coverage.info file, that has the lcov format, something like:

SF:d:\src\cygwin\home\zhaoqin\workspace\dynamorio\dynamorio.git\suite\tests\common\fib.c

DA:60,1

DA:61,1

DA:63,1

DA:64,0

DA:65,1

...

DA:92,1

end_of_record

You can use genhtml from lcov to generate the html file. There is a copy of genhtml from DynamoRIO source code at dynamorio.org.

Note, genhtml does not work with the windows path, so you might want to covert the path unix format.

Documentation about drcov is at http://dynamorio.org/docs/page_drcov.html

Let me know if you need any help.

[zhaoqin]

Also, DrCov is used by Chrome QA team for collecting coverage on Windows.

[Liuwei Lian]

Thank you! that help a lot！

[Liuwei Lian]

One more thing, I look through the generate_coverage.py script, found there is a way to do code coverage by Visual Studio Performance Tools. 

but I can't find the tool name coverage_analyzer.exe under "../third_party/coverage_analyzer/bin". it is a tools coverage .coverage file to .lcov file.

Google has plan to public coverage_analyzer.exe ?

Any Way to stats .coverage by file?

On Thursday, November 13, 2014 12:40:09 AM UTC+8, Chris Hamilton wrote:

[Chris Hamilton]

That tool didn't work with chrome as of MSVS2008, but I don't think it's been tried recently. It used to only ship with certain versions (Team Tools Edition, IIRC), but I don't know what the corner status is.