| When running the test suites 10 times about 2 times out of 10 the testWorkflowStepSetsUnsuccessfulMessage fails.
butler@jenkins:~/projects/gerrit-trigger-plugin$ cat dotest.sh
#mvn clean package -DskipTests
#581 grep elapsed *.out | sed "s,\(.*sed: \([^ ]*\).*\),\2 \1,g" | sort -n
for i in `seq 1 10`
do
time ./doit.sh 2>&1 | tee log.$i.out
done
butler@jenkins:~/projects/gerrit-trigger-plugin$ cat doit.sh
#mvn clean package -DskipTests
time mvn clean package
butler@jenkins:~/projects/gerrit-trigger-plugin$ clear
butler@jenkins:~/projects/gerrit-trigger-plugin$ cat doit.sh
#mvn clean package -DskipTests
time mvn clean package
butler@jenkins:~/projects/gerrit-trigger-plugin$ cat dotest.sh
for i in `seq 1 10`
do
time ./doit.sh 2>&1 | tee log.$i.out
done
butler@jenkins:~/projects/gerrit-trigger-plugin$ grep ERR log.*.out
log.4.out:[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 123.863 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest
log.4.out:[ERROR] testWorkflowStepSetsUnsuccessfulMessage(com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest) Time elapsed: 16.302 s <<< FAILURE!
log.4.out:[ERROR] Run 1: WorkflowTest.testWorkflowStepSetsUnsuccessfulMessage:184
log.8.out:[ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 183.403 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.ParameterModeJenkinsTest
log.8.out:[ERROR] testNameAndEmailParameterModeDefaultChangeAbandoned(com.sonyericsson.hudson.plugins.gerrit.trigger.spec.ParameterModeJenkinsTest) Time elapsed: 14.454 s <<< ERROR!
log.8.out:[ERROR] Run 1: ParameterModeJenkinsTest.testNameAndEmailParameterModeDefaultChangeAbandoned:166 » NullPointer
log.9.out:[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 120.08 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest
log.9.out:[ERROR] testWorkflowStepSetsUnsuccessfulMessage(com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest) Time elapsed: 16.366 s <<< FAILURE!
log.9.out:[ERROR] Run 1: WorkflowTest.testWorkflowStepSetsUnsuccessfulMessage:184
I also find the tests take quite a long time to run (20 minutes on a decent laptop) here are the top 10 "offenders"
// code placeholder
butler@jenkins:~/projects/gerrit-trigger-plugin$ grep elapsed log.1.out | sed "s,\(.*sed: \([^ ]*\).*\),\2 \1,g" | sort -n | tail -n 10 | sort -n -r
811.855 [INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 811.855 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.SpecGerritTriggerHudsonTest
231.665 [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 231.665 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServerHudsonTest
212.626 [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 212.626 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.playback.GerritMissedEventsFunctionalTest
199.747 [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 199.747 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.LockedDownGerritEventTest
179.986 [INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 179.986 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.ParameterModeJenkinsTest
150.987 [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 150.987 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.DuplicateGerritListenersHudsonTestCase
123.204 [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 123.204 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.project.GerritTriggerProjectHudsonTest
118.891 [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.891 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest
105.287 [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 105.287 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.actions.manual.ManualTriggerActionPermissionTest
102.878 [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.878 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.DuplicateGerritListenersPreloadedProjectHudsonTestCase
|
