Jenkins / Unity / Guthub integration

[andrew stillie]

I am  novice to using Unity and Jenkins.

I have setup a Jenkins Server on an AWS server.  This is integrated with github, such that when I commit code to the repo, it triggers jenkins to run my Unity test scripts.

However, if I introduce a bug to force the unity test to fail, the console output indicates the FAIL to that particular test, Jenkins pass the overall test as a pass.

I believe this is because Jenkins is detecting the "SUCCESS" keyword at the end of the console output.

test/TestEdmundCode.c:19:test_handleEdmundBehaviour_TestNoFalseExitsFromStateShutdown:PASS

test/TestEdmundCode.c:35:test_handleEdmundBehaviour_TestTrueExitsFromIdleState:PASS test/TestEdmundCode.c:46:test_handleEdmundBehaviour_TestNoFalseExitsFromStatePoweringUp:PASS test/TestEdmundCode.c:60:test_handleEdmundBehaviour_TestTrueExitsFromStatePoweringUp_behaviourEventButtonDownLT1000:PASS test/TestEdmundCode.c:70:test_handleEdmundBehaviour_TestTrueExitsFromStatePoweringUp_behaviourEventButtonDownGT1000LT5000:FAIL: Expected 5 Was 7 test/TestEdmundCode.c:72:test_handleEdmundBehaviour_TestTrueExitsFromStatePoweringUp_behaviourEventButtonDownGT5000:PASS test/TestEdmundCode.c:82:test_handleEdmundBehaviour_TestNoFalseExitsFrombehaviourStatePoweringDn:PASS test/TestEdmundCode.c:97:test_handleEdmundBehaviour_TestTrueExitsFrombehaviourStatePoweringDn_behaviourEventLowBattery:PASS test/TestEdmundCode.c:103:test_handleEdmundBehaviour_TestTrueExitsFrombehaviourStatePoweringDn_behaviourEventButtonDownGT2000Rleased:PASS test/TestEdmundCode.c:114:test_handleEdmundBehaviour_TestNoFalseExitsFrombehaviourStatePoweringUpLowBattery:PASS test/TestEdmundCode.c:128:test_handleEdmundBehaviour_TestTrue1ExitsFrombehaviourStatePoweringUpLowBattery_behaviourEventButtonDownLT1000:PASS test/TestEdmundCode.c:134:test_handleEdmundBehaviour_TestTrue1ExitsFrombehaviourStatePoweringUpLowBattery_behaviourEventButtonDownGT1000LT5000:PASS test/TestEdmundCode.c:140:test_handleEdmundBehaviour_TestTrue1ExitsFrombehaviourStatePoweringUpLowBattery_behaviourEventButtonDownGT5000:PASS test/TestEdmundCode.c:150:test_handleEdmundBehaviour_TestNoFalseExitsFrombehaviourStateConnectedAuthApp:PASS test/TestEdmundCode.c:166:test_handleEdmundBehaviour_TestTrueExitsFrombehaviourStateConnectedAuthApp:PASS test/TestEdmundCode.c:175:test_handleEdmundBehaviour_TestNoFalseExitsFromStateAdvertising:PASS test/TestEdmundCode.c:190:test_handleEdmundBehaviour_TestTrueExitsFromStateAdvertising_behaviourEventConnectedAuth:PASS test/TestEdmundCode.c:196:test_handleEdmundBehaviour_TestTrueExitsFromStateAdvertising_behaviourEventButtonDownGT50:PASS test/TestEdmundCode.c:206:test_handleEdmundBehaviour_TestNoFalseExitsFromConnectedUnAuthApp:PASS test/TestEdmundCode.c:222:test_handleEdmundBehaviour_TestTrueExitsFromConnectedUnAuthApp:PASS test/TestEdmundCode.c:232:test_handleEdmundBehaviour_TestNoFalseExitsFrombehaviourStateLowBattShutdown:PASS ----------------------- 21 Tests 1 Failures 0 Ignored FAIL make: [makefile:57: default] Error 1 (ignored) + echo 0 0 [Set GitHub commit status (universal)] SUCCESS on repos [] (sha:6af4ca3) with context:TestJobGit Finished: SUCCESS

How do I get Unity/Jenkins to detect that the individual test has failed.  The icing on the cake would be to somehow mark the git commit as a failure also.

Thank in advance.

[Björn Pedersen]

andrew....@googlemail.com schrieb am Freitag, 28. Juli 2023 um 14:28:33 UTC+2:

I am  novice to using Unity and Jenkins.

make: [makefile:57: default] Error 1 (ignored) + echo 0

How do I get Unity/Jenkins to detect that the individual test has failed.  The icing on the cake would be to somehow mark the git commit as a failure also.

 This the problem:  your makefile or test run script (as you did not post the complete setup it's hard to guess) should return with a non-zero status to fail the  step.

Alternativly you could use either  the JUnit Plugin (if you can produce a junit-comaptible output) or Warnings Next Generation Plugin ( which is highly customisable to parse any output) to fail the build.

 

Thank in advance.