Pitest jenkins plugin not failing builds

73 views
Skip to first unread message

Sándor Bokor

unread,
Apr 12, 2021, 12:25:20 PM4/12/21
to PIT Users
Hi,

We are using pitest for android development.

Turns out at some point jenkins stopped failing our builds when the coverage is below 100%.
Have tried to reinstall the plugin (https://plugins.jenkins.io/pitmutation/) (with cleaning all the leftover plugin data, restar between re-installs, etc) but it still passes the build.

In the logs I can clearly see that the coverage is like 98.93% but the build is marked as successful.

Our jenkins version is 2.263.1

Did anybody run into anything like this?

Thank you!

Ullrich Hafner

unread,
Apr 12, 2021, 2:24:04 PM4/12/21
to pitu...@googlegroups.com
Am 12.04.2021 um 18:25 schrieb Sándor Bokor <san...@gmail.com>:

Hi,

We are using pitest for android development.

Turns out at some point jenkins stopped failing our builds when the coverage is below 100%.
Have tried to reinstall the plugin (https://plugins.jenkins.io/pitmutation/) (with cleaning all the leftover plugin data, restar between re-installs, etc) but it still passes the build.

In the logs I can clearly see that the coverage is like 98.93% but the build is marked as successful.


What exactly does the console log in Jenkins show?

Our jenkins version is 2.263.1

Did anybody run into anything like this?

Thank you!


--
You received this message because you are subscribed to the Google Groups "PIT Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pitusers+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pitusers/2c92d5c4-7de6-4ef6-956a-7468c85640d7n%40googlegroups.com.

Sándor Bokor

unread,
Apr 12, 2021, 2:33:00 PM4/12/21
to PIT Users
The log is pretty normal and it correctly shows the calculated percentage.

BUILD SUCCESSFUL in 4m 35s 
70 actionable tasks: 69 executed, 1 up-to-date 
Looking for PIT reports in REDACTED 
Publishing mutation report: REDACTED/reports/pitest/mutations.xml 
Kill ratio is 98.933% (4359 4406) 
Finished: SUCCESS

Ullrich Hafner

unread,
Apr 12, 2021, 2:34:43 PM4/12/21
to pitu...@googlegroups.com
And how looks your job configuration? Did you correctly set up the quality gate?

Sándor Bokor

unread,
Apr 12, 2021, 2:46:04 PM4/12/21
to PIT Users
I would say yes. (This used to work and based on further investigation this broke a while ago but we were good about writing tests so we didn't notice it until now.)

Looks like can't upload a screenshot due to network security, but basically the plugin asks for the path and the Minimum kill % which is set to 100.0
Have tried it with and without Kill ratio must improve but no difference. 

Ullrich Hafner

unread,
Apr 12, 2021, 2:48:54 PM4/12/21
to pitu...@googlegroups.com
Then please create a new bug report in Jenkins issue tracker: https://issues.jenkins.io/

Reply all
Reply to author
Forward
0 new messages