Problem with karma-coverage

[amis...@gmail.com]

Hi, guys:

I write these lines because I have a problem implementing Karma with karma-coverage plugin, reporting as cobertura plugin, in Jenkins.

I have the last versions of Karma and karma-coverage and my problem is that when I run the tests with Karma in Windows I get the coverage xml report filled with the info, but when the tests are run by Jenkins the xml has not info about the coverage only source node is filled.

Anyone has the same issue? Anyone knows how I can solve it?

Thanks in advance.

[Tomás Corral Casas]

Any one can help me?

[Vojta Jína]

What is the difference between your local setup vs. jenkins?

Do you use the same karma.conf.js ? Are the paths the same?

Can you debug your jenkins test run - check if the source files are instrumented?

V.

--
You received this message because you are subscribed to the Google Groups "karma-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to karma-users...@googlegroups.com.
To post to this group, send email to karma...@googlegroups.com.
Visit this group at http://groups.google.com/group/karma-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/46e8bab3-1e2d-4969-9828-1992ac8a2868%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[Tomás Corral Casas]

I'm using Karma 0.10.4 with karma-coverage 0.1.0

Jenkins and Windows environment get the source code from a repository:

• In Windows I've it in my local hard drive "C:\code"
  
• In Jenkins is a Linux machine and it's in "/var/lib/jenkins/jobs/HarnessJS/workspace/source/"

The files seems to be instrumented as you can see below.

When I run the tests I see how each file that should be pre-processed is preceded by [pre-processor coverage]  (aaaaa,bbbbb.cccc are not the real folder names)

[preprocessor.coverage]:  [39mProcessing "/var/lib/jenkins/jobs/HarnessJS/workspace/source/aaaaa/bbbbb/cccc/public/js/internal/init.js" 

This is as my karma.conf.js looks: I use swig to replace variables.

module.exports = function(config)

{

    config.set({

        basePath: '',

        frameworks: ['jasmine'],

        files: [

            '{{ jstdadapter }}'

            {% for file in load %}

                , "{{ file }}"

            {% endfor %}

            {% for file in test %}

                , "{{ file }}"

            {% endfor %}

        ],

        exclude: [],

        preprocessors: {

            '{{ preprocessorPath }}': ['coverage']

        },

        coverageReporter: {

            type: 'cobertura',

            dir : '{{ coveragePath }}',

            file : '{{ coverageFile }}'

        },

        reporters: [ 'progress', 'junit', 'coverage' ],

        junitReporter: {

            outputFile: '{{ output }}'

        },

        port: '{{ browser_port }}',

        runnerPort: '{{ cli_port }}',

        colors: true,

        logLevel: '{{ log_level }}',

        autoWatch: false,

        browsers: ['PhantomJS'],

        captureTimeout: '{{ timeout }}',

        singleRun: true,

        reportSlowerThan: '{{ slow_test }}',

        plugins: [

            'karma-jasmine',

            'karma-chrome-launcher',

            'karma-firefox-launcher',

            'karma-junit-reporter',

            'karma-coverage',

            'karma-phantomjs-launcher'

        ]

    });

}

I get the xml report for tests output, because it runs all the tests but I get an empty xml for coverage report: (aaaa is not the real folder name)

<?xml version="1.0" ?>

<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">

<coverage lines-valid="0"  lines-covered="0"  line-rate="1"  branches-valid="0"  branches-covered="0"  branch-rate="1"  timestamp="1383519736664" complexity="0" version="0.1">

<sources>

        <source>/var/lib/jenkins/jobs/HarnessJS/workspace/source/libs/admin_tools/ci/aaaa</source>

</sources>

<packages>

</packages>

</coverage>

[Vojta Jína]

That's the thing, you are using these variables, the paths are different and that's probably the problem. Not that the paths can't be different, but your paths on Linux somehow don't work. Check out the generated karma.conf.js

Can you also check the coverage-Chrome-xxx.json file generated by the coverage? Does it contain coverage data?

It's more likely the coverage data from the browser contain slightly different paths than the reporter expects and therefore the coverage is empty.

V.

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/056c7b4a-8ff5-455e-a9e5-7ece553ba013%40googlegroups.com.

[Tomás Corral Casas]

I run the tests using PhantomJS 1.9.2 .

This is the xml that generates in Jenkins (Linux machine) 

<?xml version="1.0" ?>

<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">

<coverage lines-valid="0"  lines-covered="0"  line-rate="1"  branches-valid="0"  branches-covered="0"  branch-rate="1"  timestamp="1383519736664" complexity="0" version="0.1">

<sources>

        <source>/var/lib/jenkins/jobs/HarnessJS/workspace/source/libs/admin_tools/ci/aaaa</source>

</sources>

<packages>

</packages>

</coverage>

2013/11/4 Vojta Jína <vojta...@gmail.com>

--
You received this message because you are subscribed to a topic in the Google Groups "karma-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/karma-users/ehTrrPEipq8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to karma-users...@googlegroups.com.

To post to this group, send email to karma...@googlegroups.com.
Visit this group at http://groups.google.com/group/karma-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/CAB-zMX%3DFbN%2BDeGufnmSWyRAVH4OYEv3QwtYyHKV3ygfpYx__rw%40mail.gmail.com.

[Tomás Corral Casas]

How I can see if the files have been instrumented? What is different from the original?

[Tomás Corral Casas]

I've attached some files with the complete info:

• The console result of executing the tests with Karma in Jenkins, Linux (consoleTextJenkins.txt) 
• The console result of executing the tests with Karma in my computer, Windows 7 (consoleTextWindows.txt)
  
• The coverage xml generated in my computer, Windows 7 (coverageWindows.xml)
• The coverage xml generated in Jenkins, Linux (coverageJenkins.xml)

Could you see anything that could help me? 

[Tomás Corral Casas]

I've added in this post the console log of running the tests in Jenkins adding a console.log for instrumentedCode and it seems that is working but the report remains being empty.

:(

[Vojta Jína]

You can see the files are instrumented, by checking the source - go to the browser and open web inspector, "sources" tabs. The instrumented code will contain lots of tracing, stuff like:

if (typeof __coverage__ === 'undefined') { __coverage__ = {}; }

if (!__coverage__['./lib/minus.js']) {

   __coverage__['./lib/minus.js'] = {"path":"./lib/minus.js","s":{"1":1,"2":0},"b":{},"f":{"1":0},"fnMap":{"1":{"name":"minus","line":2,"loc":{"start":{"line":2,"column":0},"end":{"line":2,"column":21}}}},"statementMap":{"1":{"start":{"line":2,"column":0},"end":{"line":4,"column":1}},"2":{"start":{"line":3,"column":2},"end":{"line":3,"column":15}}},"branchMap":{}};

}

var __cov_0jv26E7WJSUQ72L9FnF2Nw = __coverage__['./lib/minus.js'];

function minus(a,b){__cov_0jv26E7WJSUQ72L9FnF2Nw.f['1']++;__cov_0jv26E7WJSUQ72L9FnF2Nw.s['2']++;return a-b;}

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/a92c2270-0351-4a85-b79f-143debe1d9f8%40googlegroups.com.

[Vojta Jína]

Can you try using basePath? rather than generating absolute paths?

The way you are doing it should work, but not that many people use it, so I guess there might be some bug in karma-coverage when using absolute paths.

Can you check the "coverage-Phantom...json" file ? That should be the raw coverage data from the browser where you can see the file paths.

Also, can you find your node_modules/karma-coverage/lib/reporter.js and console.log this variable:

https://github.com/karma-runner/karma-coverage/blob/master/lib/preprocessor.js#L17

This is probably lame, the preprocessor is trying to make the paths relative as that's what istanbul expects.

The istanbul reporter might only write coverage for files within the "basePath":

https://github.com/karma-runner/karma-coverage/blob/master/lib/reporter.js#L115

I need to look into this more to figure out what's the best way to fix it, but there's definitely something wrong with karma-coverage, your setup is correct I think.

V.

[Tomás Corral Casas]

Hi, Vojta!

I've changed to use basePath in Jenkins (Linux machine) but I get the same problem. I've added some extra logs to see what's happening.

Please take a look to the file. 

Thanks in advance. :)

[krots...@gmail.com]

We've got the same problem here, using Karma 0.10.4 and Karma-Coverage 0.1.1. After some digging, the issue seems to be in the coverage plugin - there is a hook in onBrowserStart which generates the coverage collector, and the onBrowserEnd hook then checks for the existence of that collector. Unfortunately, onBrowserStart is never triggered, therefore the collectors never execute.

There are reported issues:

https://github.com/karma-runner/karma-coverage/issues/30

There is a pull request that proposes a solution:

https://github.com/karma-runner/karma-coverage/pull/29

Discussion suggests that downgrading to 0.1.0 works, and that 0.1.1 is for Karma v0.11 only. I've tested the former against 0.10.4 and that seems to work, however I question the need to have the onBrowserStart hook at all, since the collector isn't ever used until the browser finishes.

Michael

[Vojta Jína]

Michael, I think you run into https://github.com/karma-runner/karma-coverage/issues/30 which should be fixed in karma-c...@0.1.2

The problem that Tomas is having is something different.

Tomas, did you log result.coverage in https://github.com/karma-runner/karma-coverage/blob/master/lib/reporter.js#L97 ? (I can't see that in your log)

It looks like for some reason the results are not added to the collector.

Can you also try "html" reporter rather than cobertura?

--

You received this message because you are subscribed to the Google Groups "karma-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to karma-users...@googlegroups.com.
To post to this group, send email to karma...@googlegroups.com.
Visit this group at http://groups.google.com/group/karma-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/ef3c6b8c-4e29-4e64-8310-c82ab97d1300%40googlegroups.com.

[Tomás Corral Casas]

I tried even with lcov but I got the same result. :|

2013/11/10 Vojta Jína <vojta...@gmail.com>

--
You received this message because you are subscribed to a topic in the Google Groups "karma-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/karma-users/ehTrrPEipq8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to karma-users...@googlegroups.com.

To post to this group, send email to karma...@googlegroups.com.
Visit this group at http://groups.google.com/group/karma-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/CAB-zMXnaZLWN8-cFDFPUa73-j1uK6W%3Dg1C1v%2BtSosdnDxfjDYA%40mail.gmail.com.

[Vojta Jína]

So can you try to log result.coverage in https://github.com/karma-runner/karma-coverage/blob/master/lib/reporter.js#L97 ? (I can't see that in your log)

It looks like for some reason the results are not added to the collector.

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/CABg9zXt_S4gy%2BsoVfsxZQSr5HVLegZir3csRJWYM%2BUZvRFrSsg%40mail.gmail.com.

[Tomás Corral Casas]

Hi, Vojta!

The log exist in the file I uploaded. 

You can find it looking for:

• ONresult
  
• Logs the result object
• ONresultCoverage
  
• Logs the result.coverage object

Nothing new. :(

2013/11/11 Vojta Jína <vojta...@gmail.com>

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/CAB-zMX%3D%2Bni%2BCHcCG1akae2JtPT_SDvspCwTO6LoqFkQTTn6m5g%40mail.gmail.com.

[Vojta Jína]

Tomas, I assumed that was log inside "onResult", which is called after each test. This is empty, however that is expected. Jasmine sends the coverage at the end of the suite.

I'm asking about https://github.com/karma-runner/karma-coverage/blob/master/lib/reporter.js#L97

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/CABg9zXuZcfj0e--U4iEi9nt06wjUe7XFW9is%2BckAOzCJ-pA6Rw%40mail.gmail.com.

[Vojta Jína]

Can you also check the browser's web inspector to see if there is any error?

Maybe some error happens when Jasmine is sending the coverage results and it fails...

Really don't know, just guessing.

V.

[Tomás Corral Casas]

I can't check the browser's web inspector because I'm using PhantomJS to launch the tests.

I've attached the file with the log, and it's in the same line you told me. :|

2013/11/12 Vojta Jína <vojta...@gmail.com>

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/CAB-zMXnqRvJsAEPv-hH1QMrNbgW8LYjMHOD%3D6BvUGyeKKkfTyA%40mail.gmail.com.

[Vojta Jína]

Ops, sorry I was sending a wrong link. I need onBrowserComplete (https://github.com/karma-runner/karma-coverage/blob/master/lib/reporter.js#L73), not onSpecComplete. Jasmine sends the coverage after the whole run is complete.

Can you try some regular browser so that you can see the console?

Or debug PhanomJS, see https://github.com/ariya/phantomjs/wiki/Troubleshooting#error-handling

I created a feature request to make this simpler with karma-phantomjs-launcher:

https://github.com/karma-runner/karma-phantomjs-launcher/issues/18

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/CABg9zXu%2BFOMNP8s6U86rq4QPzBmfzH3kKNJZU%2BiBKiQqcvcC-A%40mail.gmail.com.