Coverage reports in Clover format

[Евгений Молодкин]

Hi guys!

Is it possible to make test coverage reports in Clover XML format?

I trying to integrate Karma and Bamboo CI server. I've successfully get my test results in Bamboo by parsing test reports in JUnit format, but I have no idea how to import coverage reports into Bamboo.

Bamboo supports coverage reports in Clover and NCover formats. But Karma can produce only html, text, lcov and covertura reports.

I've looked for some kind of converter, but can not found any.

Any help would be greatly appreciated!

Thanks,

Eugene Molodkin.

[Vojta Jína]

Karma coverage plugin uses https://github.com/gotwarlost/istanbul, which can generate (AFAIK), html, lcov, or json coverage. I believe you should be able to convert the JSON to pretty much anything you want.

V.

--
You received this message because you are subscribed to the Google Groups "karma-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to karma-users...@googlegroups.com.
To post to this group, send email to karma...@googlegroups.com.
Visit this group at http://groups.google.com/group/karma-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Евгений Молодкин]

I`ve added the "Run Tests" task to Bamboo Job (type: script, inline) with the following command:

karma start my.karma.conf.js --single-run --reporters junit,progress

In the my.karma.conf.js file I`ve pointed output file for report by using:

junitReporter = {

    outputFile: 'test-reports/my.xml',

    suite: 'My Suite'

};

Then I just add task "JUnit Parser" in the Final tasks block, and specify the directory with my JUnit reports, e.g. **/test-reports/*.xml

Hope this helps :)

2013/5/1 <seba....@gmail.com>

I've got the same issue.

Actually how did you get your test results to parsed by Bamboo in Junit format?

--
You received this message because you are subscribed to a topic in the Google Groups "karma-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/karma-users/yww1YRdMi_s/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to karma-users...@googlegroups.com.

To post to this group, send email to karma...@googlegroups.com.
Visit this group at http://groups.google.com/group/karma-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
С уважением,

Евгений Молодкин

[sant...@gmail.com]

Worked great Thanks!

[bix...@gmail.com]

I ran into a similar scenario at my workplace, and to address the issue we implemented a reporter for Istanbul (karma-coverage uses this) that produces Clover output. (lib/report/clover.js)

Our implementation can be found here: https://github.com/seeq12/istanbul/tree/clover-report

The implementation is cloned and heavily derived from the cobertura.js file since both Cobertura and Clover reports are in XML. We used a sample clover.xml output as a reference and changed the cobertura code to match it as closely as possible.

Note: For our implementation purposes, we have made it such that Bamboo will be able to correctly parse the code coverage metrics. We have not fully tested the validity of the clover output as a whole. Through troubleshooting, we concluded that Bamboo will look in to the first <metrics> tag for all the code coverage metrics, and our implementation has included the attributes that Bamboo is looking for.

You will need to package.JSON to include our forked version of karma-coverage as follows:

  "devDependencies": {

    "karma-coverage": "git://github.com/seeq12/karma-coverage.git#clover-report",

  }, 

This will pull in our version of Istanbul. (We will create a pull request to propose that it get pulled into the gotwarlost/istanbul repo.)

After that, add the ‘clover’ report type in your karma.conf.js file to produce an output:

    coverageReporter: {

      reporters:[

        {type: 'html', dir:'coverage/'},

        {type: 'clover', dir:'<your_directory>'},

      ],

    },

Setting up Bamboo to parse your file:

1.  On the Plan’s Stage that includes running ‘grunt’ (or whatever you are using to run karma-coverage and istanbul), click the Miscellaneous tab.

2.  Check “Use Clover to collect Code Coverage for this build.”

3.  Click the “Clover is already integrated into this build…” radio button.

4.  For “Clover XML Location”, specify a file location. On our system, I use “app/coverage/**/clover.xml” which goes into the “app/coverage” directory and then searches for all clover.xml files at that level or below. You can refine this if needed for your system…

There seems to be a bug in Bamboo whereby the “Clover” tab is not visible on individual builds in some scenarios. When this happens, I go to the Plan level, select the branch I want, and then click on the “Clover” menu item (which always seems to be there). That page then contains a link to “View latest clover information”, which takes me to a build results page that DOES include the Clover tab. You can then click on the little build navigator to the left of the run button to see the Clover tab for other builds.

The other thing is that you’ll notice the following error at the end of the build logs:

simple  27-Jan-2014 16:51:12   Publishing an artifact: Clover Report (System)

error   27-Jan-2014 16:51:12   Unable to publish artifact [Clover Report (System)]: the source directory /home/bamboo/bamboo-agent-home/xml-data/build-dir/CRAB-WC1-JOB1/target/site/clover does not exist.

simple  27-Jan-2014 16:51:12   Finished publishing of artifact Job artifact: [Clover Report (System)], pattern: [**/*.*] anchored at: [target/site/clover] in 0s

You have to ignore this error. If you try copying Clover.xml to ‘target/site/clover/clover.xml’ everything will fall apart… Bamboo will stop parsing your XML file correctly and it won’t track metrics. I have no idea why.

This may not be the perfect solution but it has certainly gotten the job done for us. Hopefully it helps!

[Vojta Jína]

Nice, thanks for sharing!

--

You received this message because you are subscribed to the Google Groups "karma-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to karma-users...@googlegroups.com.

To post to this group, send email to karma...@googlegroups.com.

Visit this group at http://groups.google.com/group/karma-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/b62378e1-9bc5-4703-aa91-690a2c05a945%40googlegroups.com.

[dme...@jut.io]

Any luck getting this back upstream to gotwarlost/istanbul repo? I'd love to use your clover reporter, ideally without running a forked version. We use istanbul without karma. Thanks!

[bix...@gmail.com]

Looks like the pull request has been merged recently:

https://github.com/gotwarlost/istanbul/pull/145

Cheers

[mparfi...@atlassian.com]

Hi,

It'd like to say that you did really great job implementing this converter. It's a good stuff and we even use it internally in Atlassian :-)
 

Through troubleshooting, we concluded that Bamboo will look in to the first <metrics> tag for all the code coverage metrics, and our implementation has included the attributes that Bamboo is looking for.

I confirm. A Bamboo Clover Plugin looks for the <coverage>/<project>/<metrics> tag in XML file. It ignores rest of the file.

There seems to be a bug in Bamboo whereby the “Clover” tab is not visible on individual builds in some scenarios.

We'll it's a missing feature actually. The "Clover" tab is available for the Plan Summary and for the Job Summary pages. There's a feature request to add it for a Build Summary as well: https://jira.atlassian.com/browse/BAM-12852

The other thing is that you’ll notice the following error at the end of the build logs:

simple  27-Jan-2014 16:51:12   Publishing an artifact: Clover Report (System)

error   27-Jan-2014 16:51:12   Unable to publish artifact [Clover Report (System)]: the source directory /home/bamboo/bamboo-agent-home/xml-data/build-dir/CRAB-WC1-JOB1/target/site/clover does not exist.

simple  27-Jan-2014 16:51:12   Finished publishing of artifact Job artifact: [Clover Report (System)], pattern: [**/*.*] anchored at: [target/site/clover] in 0s

When automatic Clover integration is enabled, Clover adds a "Clover Report (System)" artifact. As you don't generate any HTML report inside target/site/clover, deleting this artifact from the "Artifacts" tab should fix this error.

Cheers
Marek

[zzzla...@gmail.com]

Thank you so much for sharing these detailed instructions.

It works for me now!

[jay...@lightshipworks.com]

This setup helped me greatly! We now have coverage through a Gulp task running Karma and it correctly reports executed tests (karma-junit-reporter) and clover statistics.

A note about the warning, it seems that turning on Clover adds an Artifact collection which is why you got the error:

simple  27-Jan-2014 16:51:12   Finished publishing of artifact Job artifact: [Clover Report (System)], pattern: [**/*.*] anchored at: [target/site/clover] in 0s

If you go in to the job configuration, you can remove the Clover artifact.