Coverage failure for recipe_modules/chromium_tests/api.py

[Michael McGreevy]

I have a CL (chromium-review.googlesource.com/c/490592) for which the presubmit is failing due to missing coverage.  It looks like it is affected by the recently submitted https://chromium-review.googlesource.com/c/496906 , because when I restore the "DISABLE_STRICT_COVERAGE = True" line (removed in 496906), my CL passes the presubmit again.

I am having trouble understanding, however, how I am missing coverage in the first place.  When I run

recipes.py test train 

I get 

Name                                                                                                        Stmts   Miss  Cover   Missing

-----------------------------------------------------------------------------------------------------------------------------------------

<snip>/src/chrome_infra/build/scripts/slave/recipe_modules/chromium_tests/api.py     422      2    99%   1046, 1068

-----------------------------------------------------------------------------------------------------------------------------------------

TOTAL                                                                                                       19272      2    99%

but if I add a print statement at e.g. line 1068 and run 'recipes.py test train' again, I can see that the print statement is sometimes executed, but I still get a warning about that line causing missing coverage.  i.e. I get:

 

Name                                                                                                        Stmts   Miss  Cover   Missing

-----------------------------------------------------------------------------------------------------------------------------------------

<snip>/src/chrome_infra/build/scripts/slave/recipe_modules/chromium_tests/api.py     423      3    99%   1046, 1068-1069

-----------------------------------------------------------------------------------------------------------------------------------------

TOTAL  

Can anyone give me some guidance about how to debug this supposed missing coverage?

[Paweł Hajdan, Jr.]

Sure.

From the original announcement:

"You may see coverage failures for new code you might have expected to be covered by recipe tests or tests in other recipe modules. Under strict coverage, these no longer count."

I believe that's exactly what you're running into. I'd recommend applying the following part of the announcement:

"The solution is to add test coverage in chromium_tests' "tests" subdirectory."

The CL (as of patchset 8) only modifies api.py inside chromium_tests recipe module and relies on chromium_trybot.py for coverage.

Since the CL modifies _trybot_steps_internal, could you move the tests to https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/tests/api/trybot_steps.py?q=trybot_steps+package:%5Echromium$&dr=C&l=16 ?

Feel free to ping me on IM if you have further questions.

Paweł