Importing Unit Test Execution Report for JS project

[praum...@gmail.com]

Hi Sonarqube,

As per the online documentation, "http://docs.sonarqube.org/display/PLUG/JavaScript+Unit+Tests+Execution+Reports+Import" it says "Importing js-test-driver reports is not supported since version 2.10: the js-test-driver project does not seem to be maintained anymore." 

Does this mean importing unit test execution report is not supported at all in 2.1 version. Is there any work around for it inorder for us to display unit test execution report.

Also I am confused which one is the correct property. I have seen both of them online.

sonar.javascript.jstestdriver.reportsPath

sonar.javascript.jstest.reportsPath

[G. Ann Campbell]

Hi,

The workaround, as documented in the deprecation notice, is to use the Generic Test Coverage plugin.

Ann

[umesh prajapati]

Hi Campbell,

Thank you for the quick response but  Generic Test Coverage plugin does not provide Unit Test Result Report Execution and just coverage report execution.

[umesh prajapati]

Hi Campbell,

I have tried using sonar.genericcoverage.unitTestReportPaths for providing the path to my unit test report execution xml file but I still dont see the unit test report in sonar dashboard where it displays the metrics Total number of Test Executed, Failed, Passed, Error and Skipped. I only see code coverage Report.

XML Report Format: Using mocha-sonar-reporter [ No Luck ]

<testsuite name="Mocha Tests" tests="11" failures="0" errors="0" skipped="0" timestamp="Tue, 31 May 2016 20:48:23 GMT" time="3.046">

<testcase classname="Test" name="validate access token should fail to delete when user is not authorized" time="0.242"/>

<testcase classname="Test" name="validate access token should fail to delete when token is not supplied" time="0.007"/>

<testcase classname="Test" name="validate access token should fail to delete when token is invalid" time="0.244"/>

<testcase classname="Test" name="validate access token should succeed to delete when token and headers are valid" time="0.49"/>

<testcase classname="Test" name="validate access token should succeed validation when token and request are valid" time="0.275"/>

<testcase classname="Test" name="validate access token should fail validation when event has invalid Authorization token" time="0.25"/>

<testcase classname="Test" name="validate access token should fail validation when event has no header or Authorization token" time="0.016"/>

<testcase classname="Test" name="validate access token should fail validation when event header has no Authorization token" time="0.012"/>

<testcase classname="Test" name="validate access token should fail validation when request is invalid" time="0.252"/>

<testcase classname="Test" name="validate access token should fail validation when Authorization header is empty" time="0.251"/>

<testcase classname="Test" name="validate access token should fail validation and delete token when token has expired" time="0.25"/>

</testsuite>

XML Report Format: Using mocha-junit-reporter [ No Luck ]

<?xml version="1.0" encoding="UTF-8"?>

<testsuites name="Mocha Tests" time="1.853" tests="11" failures="0">

  <testsuite name="Root Suite" timestamp="2016-05-31T21:28:21" tests="0" failures="0" time="0">

  </testsuite>

  <testsuite name="validate access token" timestamp="2016-05-31T21:28:21" tests="4" failures="0" time="0.796">

    <testcase name="Root Suite validate access token should fail to delete when user is not authorized" time="0.201" classname="should fail to delete when user is not authorized">

    </testcase>

    <testcase name="Root Suite validate access token should fail to delete when token is not supplied" time="0.005" classname="should fail to delete when token is not supplied">

    </testcase>

    <testcase name="Root Suite validate access token should fail to delete when token is invalid" time="0.184" classname="should fail to delete when token is invalid">

    </testcase>

    <testcase name="Root Suite validate access token should succeed to delete when token and headers are valid" time="0.406" classname="should succeed to delete when token and headers are valid">

    </testcase>

  </testsuite>

  <testsuite name="validate access token" timestamp="2016-05-31T21:28:22" tests="7" failures="0" time="1.057">

    <testcase name="Root Suite validate access token should succeed validation when token and request are valid" time="0.2" classname="should succeed validation when token and request are valid">

    </testcase>

    <testcase name="Root Suite validate access token should fail validation when event has invalid Authorization token" time="0.203" classname="should fail validation when event has invalid Authorization token">

    </testcase>

    <testcase name="Root Suite validate access token should fail validation when event has no header or Authorization token" time="0.008" classname="should fail validation when event has no header or Authorization token">

    </testcase>

    <testcase name="Root Suite validate access token should fail validation when event header has no Authorization token" time="0.008" classname="should fail validation when event header has no Authorization token">

    </testcase>

    <testcase name="Root Suite validate access token should fail validation when request is invalid" time="0.208" classname="should fail validation when request is invalid">

    </testcase>

    <testcase name="Root Suite validate access token should fail validation when Authorization header is empty" time="0.219" classname="should fail validation when Authorization header is empty">

    </testcase>

    <testcase name="Root Suite validate access token should fail validation and delete token when token has expired" time="0.211" classname="should fail validation and delete token when token has expired">

    </testcase>

  </testsuite>

</testsuites>

Property	Example	Description
sonar.genericcoverage.reportPaths	report1.xml, report2.xml	Comma separated paths to the Coverage by UT Reports
sonar.genericcoverage.itReportPaths	it_report.xml	Comma separated paths to the Coverage by IT Reports
sonar.genericcoverage.unitTestReportPaths	ut_report.xml	Comma separated paths to the Unit Tests Execution Results Report

[G. Ann Campbell]

Hi,

Have you tried converting your file to the Generic Coverage format?

Ann

---

G. Ann CAMPBELL | SonarSource

Product Owner

http://sonarsource.com

[michael.van...@gmail.com]

sorry; but the junit results produced by karma cannot seem to be converted to the required SonarQube format because they internally lack the full path to the testfile

also see

https://github.com/tornaia/karma-sonarqube-unit-reporter and it's current issues

there are multiple karma sonarqube junit reporters attempts but they all fail

is it possible to re-introduce somekind of Javascript junit report parser (like the deprecated one) because now there's no out-of-the-box working combination between SonarQube and Karma in regarding the test results

[Pierre-Yves Nicolas]

That's not in our current plans.

Unlike coverage, test results are not a primary target for SonarQube.

Regards,

Pierre-Yves

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/3bdff6a6-e3ed-493f-8cd3-836102482bc5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Pierre-Yves Nicolas]

Hi Michael,

Can you please elaborate about using the number of tests as a quality indicator?

Would you read it at project level of for each individual test file?

Thanks,

Pierre-Yves

On 6 September 2016 at 09:15, Michael van de Giessen <michael.van.de.giessen@gmail.com> wrote:

Hi Pierre-Yves,

That's weird, test results (at least the test-count, and maybe ignored test count) are also an indicator of improving or degrading quality; in a setting where coverage remains somewhat equal a increase or decrease in the amount of tests can be a indicator. Perhaps it is possible to extract the test-count from the coverage result?

-- 
Michael van de Giessen

Op 6 september 2016 bij 09:11:44, Pierre-Yves Nicolas (pierre-yves.nicolas@sonarsource.com) schreef:

That's not in our current plans.

Unlike coverage, test results are not a primary target for SonarQube.

Regards,

Pierre-Yves

[Michael van de Giessen]

Hi Pierre-Yves,

The quality indicator is at a project level; on a project in support-phase it's common that the coverage of the units tests remain equal but the number of tests increases due to tests being built per-ticket or integration tests testing the same code as unit tests. On the other hand tests being @Ignored because of quick-fixes should introduce technical debt. 

Solely using test-coverage as an indicator might even provoke less-quality code when trying to get to the 100% (for example creating big integration tests for large coverage).

Michael van de Giessen

Op 6 september 2016 bij 10:23:04, Pierre-Yves Nicolas (pierre-yv...@sonarsource.com) schreef:

[Freddy Mallet]

Hello Michael,

We should be careful about the scope of this discussion :

• It's not about the way to use code coverage information to increase (or not) the quality of the source code
• It's not about the tracking of @Ignored/@Skipped unit tests because obviously SonarQube must provide some rules to track those quality issues located in the unit test source code
• So at the end it's only about three needs :
• Tracking the number of failing/in error tests -> obviously this must be done in the CI engine and so this is useless to do that in SonarQube
• Tracking the total amount of times required to execute unit tests -> no sure that SonarQube is the best place to monitor and profile the performances of the tests
• Tracking the total number of tests -> Might be perhaps useful to quickly evaluate how "unit" are your tests. So for instance if you have 80% of code coverage for 20'000 lines of code and with 10 unit tests, that means that those tests are big integration tests. But not sure that this remaining use case deserves the configuration effort to import unit test reports.

Cheers

Freddy

On Tue, Sep 6, 2016 at 2:02 PM Michael van de Giessen <michael.van...@gmail.com> wrote:

Hi Pierre-Yves,

The quality indicator is at a project level; on a project in support-phase it's common that the coverage of the units tests remain equal but the number of tests increases due to tests being built per-ticket or integration tests testing the same code as unit tests. On the other hand tests being @Ignored because of quick-fixes should introduce technical debt. 

Solely using test-coverage as an indicator might even provoke less-quality code when trying to get to the 100% (for example creating big integration tests for large coverage).

Michael van de Giessen

Op 6 september 2016 bij 10:23:04, Pierre-Yves Nicolas (pierre-yv...@sonarsource.com) schreef:

Hi Michael,

Can you please elaborate about using the number of tests as a quality indicator?

Would you read it at project level of for each individual test file?

Thanks,

Pierre-Yves

On 6 September 2016 at 09:15, Michael van de Giessen <michael.van...@gmail.com> wrote:

Hi Pierre-Yves,

That's weird, test results (at least the test-count, and maybe ignored test count) are also an indicator of improving or degrading quality; in a setting where coverage remains somewhat equal a increase or decrease in the amount of tests can be a indicator. Perhaps it is possible to extract the test-count from the coverage result?

-- 
Michael van de Giessen

Op 6 september 2016 bij 09:11:44, Pierre-Yves Nicolas (pierre-yv...@sonarsource.com) schreef:

That's not in our current plans.

Unlike coverage, test results are not a primary target for SonarQube.

Regards,

Pierre-Yves

On 2 September 2016 at 09:32, <michael.van...@gmail.com> wrote:

sorry; but the junit results produced by karma cannot seem to be converted to the required SonarQube format because they internally lack the full path to the testfile

also see

https://github.com/tornaia/karma-sonarqube-unit-reporter and it's current issues

there are multiple karma sonarqube junit reporters attempts but they all fail

is it possible to re-introduce somekind of Javascript junit report parser (like the deprecated one) because now there's no out-of-the-box working combination between SonarQube and Karma in regarding the test results

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/3bdff6a6-e3ed-493f-8cd3-836102482bc5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/CA%2Bj%3DmR%3D8aC0OcYqNiXeE3GENtiKJcFGwzGPsmhOYpwE0WdSKjQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Freddy MALLET | SonarSource

Product Director & Co-Founder
http://sonarsource.com