| When the same testsuite appears in multiple files with the same timestamp, only the first testsuite is evaluated, any testcases in following files are silently ignored. I attached two test report XML files with typical output of our build runs, yet simplified as much as possible. As you can see, both files contain a different test case of the same testsuite "TestFoo". If the test execution is very fast it can happen that both the testsuites have the same timestamp, in which case every testcase but the first is not included in the report, as you can see on the attached screenshot. The expected behaviour is, that all the testcases are included in the report and the stage fails if any of the testcases failed. I consider this a major bug as it allows failed tests to be ignored and in the consequence broken software can be reported as "successfull" when it isn't. |
