Newbie: TAP Plugin && Matrix Builds

[Evan Van Dyke]

First time Jenkins development, so my apologies for what is probably a newbie question...   I am trying to attack JENKINS-22016, in which the TAP results are not published correctly for a Matrix project configuration.

I've found the resolution for JENKINS-12205, which solved a similar issue for XUnit.     It appears the issue is with the Dashboard View's "summarizeJob" implementation (see below).   The TAP plugin isn't publishing a generic hudson.tasks.test.TestResultProjectAction, and instead defines its own ProjectAction class.

What I'm struggling with is how to add a new instance of the TestResultProjectAction to the TAP plugin.  

The plugin is already publishing a ProjectAction class, and it implements a different API than the generic Hudson one.  

I'm hesitant to just replace the one with the other... and even more hesitant to add another special-case to the dashboard plugin to handle TAP.

Any thoughts/suggestions from those familiar with the Jenkins Core? What's the best path forward here?

I see three potential paths forward:

1) Replace the TAP Plugin's ProjectAction class with the generic (probably best long-term)

2) Descend the TAP Plugin's ProjectAction class from the generic

3) Add a new special-case breakout in the dashboard view

private static void summarizeJob(Job job, TestResultSummary summary) {

      boolean addBlank = true;

      TestResultProjectAction testResults = job.getAction(TestResultProjectAction.class);

 

      if (testResults != null) {

        AbstractTestResultAction tra = testResults.getLastTestResultAction();

 

         if (tra != null) {

            addBlank = false;

            summary.addTestResult(new TestResult(job, tra.getTotalCount(), tra.getFailCount(), tra.getSkipCount()));

        }

      } else {

        SurefireAggregatedReport surefireTestResults = job.getAction(SurefireAggregatedReport.class);

        if (surefireTestResults != null) {

            addBlank = false;

            summary.addTestResult(new TestResult(job, surefireTestResults.getTotalCount(), surefireTestResults.getFailCount(), surefireTestResults.getSkipCount()));

        }

       }

[Bruno P. Kinoshita]

Hi Evan!

>First time Jenkins development, so my apologies for what is probably a newbie question.

No need to apologise for that, that's an excellent question. Thanks for posting it to the mailing list. This is one of the issues that I want to fix in tap-plugin in the next releases (another one is related to performance).

When the plug-in was created, it was implemented with custom views/actions. Some time later users wanted to be able to use the same default test UI (i.e. the same graphs and views generated with JUnit tests). So we tried to reproduce what was done in (IIRC) testng-plugin.

>I'm hesitant to just replace the one with the other... and even more hesitant to add another special-case to the dashboard plugin to handle TAP.

Let's try to tackle JENKINS-22016 first by changing only tap-plugin. I'll have some spare time this weekend (forecast says it's going to rain here :-) so I can take a look at the API for Matrix projects (never used that) and see if we can simply add a new action, or go with your 1) or 2) options, keeping backward compatibility in tap-plugin and not changing core or other plug-ins.

If you have anything now in a fork, feel free to submit a pull request any time, as we can review the changes, and I can help testing it locally too. And then drop or update the pull request. Or if you want to have a quick chat about parts of the plug-in code, I'm normally lurking in #jenkins as kinow.

Cheers

Bruno

---

From: Evan Van Dyke <eva...@gmail.com>
To: Jenkins Developers <jenkin...@googlegroups.com>
Sent: Wednesday, 4 May 2016 10:28 AM
Subject: Newbie: TAP Plugin && Matrix Builds

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/abffee1f-2708-4eeb-ac72-50c110c2eeb5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Evan Van Dyke]

Hey, thanks for the quick reply!

No, I have nothing in a fork (yet), although I'm likely to try poking around.    Trying to get work to fund my time, since I'm doing this to fix things up for them...  but trying to get maven and get working behind all our firewalls is a bear.   Glad that this is on your radar already; definitely something that will make our team happy.  If you get to it first, I will not be upset by any means.    #1 need is to have all the individual test results show up on the Dashboard view.  Thanks to the -12205 fix, it now shows each of the matrix sub-jobs; they just claim they have 0 tests in them (though the test trend graph works just fine).

First one to the finish line wins...  and the sooner the better!

Again, thanks for the quick reply.

[Evan Van Dyke]

I can definitely confirm that inheriting the AbstractTapProjectAction class from the generic TestProjectAction class makes the Dashboard View's number of tests show up correctly (yay!).   However, it then breaks the "test trend" graphs, so there's definitely some interesting interaction between the current code and the generic plugin.

Still, looks like this is on the right track...

[Evan Van Dyke]

Okay...   Update on Git which fixes 1/2 of the problems (the Dashboard view now shows test statistics correctly).

Still need to determine how to get the aggregate test chart on the Matrix job main page.

Submitting a pull request with this part of the fix from:  https://github.com/evandy0/tap-plugin

Please let me know what you think... you know your plugin best, and I totally just did a hack-job here (but a working hack-job).

[Evan Van Dyke]

And...  the 2nd half has now been added as well.   All looking good on my local Jenkins server.

Let me know... this should fully address the defect if accepted.