| We started to use retry policy while running our tests, because some tests are flaky. We use https://github.com/gradle/test-retry-gradle-plugin for that. The thing is, that a test might appear N times in junit XML report, where N is number of retries. For example 2 times as a "failed" test, and 1 time as "passes". Such test ideally should be marked as flaky and depending on a junit configuration it should not mark the build as unstable/failed. Current behaviour: It's enough if 1 run of a given test fails and that will make junit plugin to mark build as unstable. Even though there are also positive passes for the same test (separate records in xml). Expected behaviour: Group tests by name, if at least one run is "passes", mark the test as: a) ideal solution: flaky one b) good enough: "passes" one. There is some another junit jenkins plugin that seems to support flaky tests, but is no longer supported (4 years now) and does not work anymore. It would be nice to add such feature in junit plugin, as discovering flaky tests seems to become a standard. |
