Can I tell whether a step succeeded beyond its effect on the build result?
26 views
Skip to first unread message
Nate Rook
Nov 24, 2014, 6:58:27 PM11/24/14
to jenkin...@googlegroups.com
Hello! I'm working on a project to display the results of Jenkins builds in an external dashboard, and I've run into a fun conceptual problem.
I'd like to tell if a BuildStep had a good or bad outcome. A 'good' outcome is one in which the step executed as expected, while a 'bad' outcome is one in which somebody should probably take a look at the build to see what's wrong with it. For instance, if an Execute Shell command exited with return code 0, that's good; if it exited with return code 1, that's bad. Similarly, if a "Publish JUnit test result report" reports all tests passed, that's good, and if some tests failed, that's bad.
Currently I'm guessing the outcome of steps by heuristics: If the BuildStep made the build result worse, as reported by AbstractBuild#getResult, then we say the step had a bad outcome. If the BuildStep requests the step abort (that is, from our StepNotifier we see canContinue = false), the step had a bad outcome. And so on.
But such heuristics will still be wrong sometimes. Say one post-build action notices a test failed, and marks the build as unstable. Then a second post-build action notices another test failed. Since the build was already unstable, my plugin can't tell that the second action had a bad outcome; it didn't make the build's result any worse.
Of course, I can add more heuristics. (For instance, here I can hook into test result reporting, and say an action had a bad outcome if it reported some failing tests.) But I'll always guess wrong in some case. Is there a more general way to tell whether a step had a good or bad outcome?
I'd like to tell if a BuildStep had a good or bad outcome. A 'good' outcome is one in which the step executed as expected, while a 'bad' outcome is one in which somebody should probably take a look at the build to see what's wrong with it. For instance, if an Execute Shell command exited with return code 0, that's good; if it exited with return code 1, that's bad. Similarly, if a "Publish JUnit test result report" reports all tests passed, that's good, and if some tests failed, that's bad.
Currently I'm guessing the outcome of steps by heuristics: If the BuildStep made the build result worse, as reported by AbstractBuild#getResult, then we say the step had a bad outcome. If the BuildStep requests the step abort (that is, from our StepNotifier we see canContinue = false), the step had a bad outcome. And so on.
But such heuristics will still be wrong sometimes. Say one post-build action notices a test failed, and marks the build as unstable. Then a second post-build action notices another test failed. Since the build was already unstable, my plugin can't tell that the second action had a bad outcome; it didn't make the build's result any worse.
Of course, I can add more heuristics. (For instance, here I can hook into test result reporting, and say an action had a bad outcome if it reported some failing tests.) But I'll always guess wrong in some case. Is there a more general way to tell whether a step had a good or bad outcome?
Ulli Hafner
Nov 25, 2014, 6:32:43 AM11/25/14
to jenkin...@googlegroups.com
I don’t think that you can generalize it. You can have a look at the CI game plug-in which does a similar thing: for each build step or post build step an individual class handles the outcome and adds points to the overall result.
--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/a1578804-6fa2-457e-b17b-14a9522e3609%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Nate Rook
Dec 1, 2014, 1:30:42 PM12/1/14
to jenkin...@googlegroups.com
Ah well, I'll keep using my silly heuristics then :) Thanks for the pointer to the CI game plugin, too.
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/NHQCjYsQ6Ls/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/8E7375D6-5389-4916-BE5C-A1F77BEFF65B%40gmail.com.
Reply all
Reply to author
Forward
0 new messages