Coverage failure should be more explicit

89 views
Skip to first unread message

Aaron Gunderson

unread,
Dec 7, 2021, 11:02:58 AM12/7/21
to elixir-lang-core
We are running elixir:latest in CI and hit this ambiguous problem with the new change to coverage thresholds reporting an error:

Changelog notes:
[mix test] Exit with a status of 3 when coverage falls below threshold

Reading CI it was unclear what was failing as the only indication of an error is the red on the coverage percentage which obviously isn't visible in this CI configuration. I think it would be much better to have an explicit failure line like:

```
-----------|--------------------------
    63.13% | Total
    
Coverage failed, threshold not met:
  Coverage:  63.13%
  Threshold: 75%
 
See https://hexdocs.pm/mix/1.13.0/Mix.Tasks.Test.html#module-coverage

Generated HTML coverage results in "cover" directory
```

Thoughts?

coverage-failure-2.png
coverage-failure.png

José Valim

unread,
Dec 7, 2021, 12:55:39 PM12/7/21
to elixir-lang-core
I like the additional input on why it failed, a PR is definitely welcome.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/2c1ce6cc-c0bd-4f44-a5ee-05d0f0c9c880n%40googlegroups.com.

Aaron Gunderson

unread,
Dec 8, 2021, 11:27:09 AM12/8/21
to elixir-lang-core
Thank you José!

For context closure we resolved this in https://github.com/elixir-lang/elixir/pull/11457
Reply all
Reply to author
Forward
0 new messages