Single run of tests ends up re-running tests many times

[Rob Arnold]

Awesome! You are blowing those tests out of the water!

In all seriousness. I've seen that too, but my configuration is a one time run - but invoked continuously from grunt.

Seems like something from the previous instance is not getting closed and multiple testacular servers are responding. May be related to yours.

Would like to hear any thoughts as well.

[Vojta Jína]

Woow. This is really weird.

Can you run it with --log-level debug (I'm just curious about the browsers connecting/disconnecting).

Does it happen always ? Or just sometimes ? Can you try to run only subset of tests, maybe there is just a single test causing it (check async tests first, if you have some).

Can you go to http://localhost:9876/debug.html and pop up the web inspector ? It should log each test executed, isn't there a single unit test that's being executed many times ?

V.

--
You received this message because you are subscribed to the Google Groups "karma-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to karma-users...@googlegroups.com.
To post to this group, send email to karma...@googlegroups.com.
Visit this group at http://groups.google.com/group/karma-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[Scott Mathson]

So this ended up getting worse and worse as our test suite grew until it wasn't possible to ignore it anymore and I was finally able to track it down by excluding subsets of the tests until the problem went away and then looking for something odd about those tests. Turned out to be tests that were executing code that called $window.location.reload() which ended up restarting the test suite run. Instead of mocking out $window.location like we should have been, we were just spying on reload() to assert that it was called. Wrapping the reload call in a service and mocking it out for the tests fixed the problem.

I guess the actual reload call was taking a little while to execute so when our test suite was still small it didn't cause any problems because the test execution would finish before the reload did it's thing, but as our test suite grew we were essentially racing for the tests to finish before the reload() call reset them to start from the beginning. So turned out to be operator error and not a karma issue, but maybe this will help someone experiencing similar issues.

Thanks for responding Vojta!

[Vojta Jína]

This sucks, we need to fix it. There's already issue for that https://github.com/karma-runner/karma/issues/27

You should not be doing it, however, if you do, Karma should scream at you, not behave weird like this.

V.

[Jayr Motta]

I've started a project recently with the yeoman's angular-generator and my tests are being run multiple times. However, it doesn't seem to be the reasons mentioned above.

I'm running node v0.10.24, phantomjs 1.9.1 and the following package.json:

{

  "name": "foobar",

  "version": "0.0.0",

  "dependencies": {},

  "devDependencies": {

    "grunt": "~0.4.1",

    "grunt-autoprefixer": "~0.4.0",

    "grunt-bower-install": "~1.0.0",

    "grunt-concurrent": "~0.5.0",

    "grunt-contrib-clean": "~0.5.0",

    "grunt-contrib-concat": "~0.3.0",

    "grunt-contrib-connect": "~0.5.0",

    "grunt-contrib-copy": "~0.4.1",

    "grunt-contrib-cssmin": "~0.7.0",

    "grunt-contrib-htmlmin": "~0.1.3",

    "grunt-contrib-imagemin": "~0.3.0",

    "grunt-contrib-jshint": "~0.7.1",

    "grunt-contrib-uglify": "~0.2.0",

    "grunt-contrib-watch": "~0.5.2",

    "grunt-google-cdn": "~0.4.0",

    "grunt-newer": "~0.6.1",

    "grunt-ngmin": "~0.0.2",

    "grunt-rev": "~0.1.0",

    "grunt-svgmin": "~0.2.0",

    "grunt-usemin": "~2.0.0",

    "grunt-ng-constant": "~0.5.0",

    "jshint-stylish": "~0.1.3",

    "load-grunt-tasks": "~0.4.0",

    "time-grunt": "~0.2.1",

    "karma-ng-scenario": "~0.1.0",

    "grunt-karma": "~0.8.2",

    "karma": "~0.12.9",

    "karma-coverage": "~0.2.1",

    "karma-phantomjs-launcher": "~0.1.4",

    "karma-junit-reporter": "~0.2.2",

    "grunt-stubby": "~0.1.4",

    "karma-jasmine": "~0.1.5",

    "karma-html-reporter": "~0.2.3",

    "grunt-connect-proxy": "~0.1.10"

  },

  "engines": {

    "node": ">=0.10.0"

  },

  "scripts": {

    "test": "grunt test"

  }

}

Could you guys help me to discover the cause and solution to this problem. Since I'm a little floating on where to start digging a tip of how I could debug would be a good start.

Thanks in advance!

[Vojta Jína]

Can you try a regular browser (eg. Chrome, FF) so that you can better debug what's going on there.

Also run karma with --log-level debug and send us the output.

How do you trigger the run? Are you using autoWatch?

Visit this group at http://groups.google.com/group/karma-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/fcbe2e7c-b8a4-411a-a625-76377e2d41ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jesse]

I'm also experiencing my unit tests running multiple times and it is reproduceable. 

It occurs with karma-jasmine ~0.2.0 but when I switch to karma-jasmine ~0.1.0 it is does not occur. All other variables are held constant.

This suggests to me it is something in the karma-jasmine files for 2.0? ( index.js, adapter.js, boot.js, jasmine.js ).

Logging statements: 

[Index.js] is initialized

Running "karma:unit" (karma) task

karma task

karma task:server.start

DEBUG [plugin]: Loading karma-* from ...

INFO [karma]: Karma v0.12.22 server start

INFO [launcher]: Starting browser PhantomJS

DEBUG [web-server]: serving: B:\workspaces\webapps\app1\node_modules\karma\static/client.html

DEBUG [web-server]: serving: B:\workspaces\webapps\app1\node_modules\karma\static/karma.js

DEBUG [web-server]: serving (cached): B:\workspaces\webapps\app1\node_modules\karma\static/context.html

DEBUG [web-server]: serving (cached): B:/workspaces/app1/webapps/node_modules/karma-jasmine/lib/jasmine.js

DEBUG [web-server]: serving (cached): B:/workspaces/app1/webapps/node_modules/karma-jasmine/lib/boot.js

DEBUG [web-server]: serving (cached): B:/workspaces/app1/webapps/node_modules/karma-jasmine/lib/adapter.js

[...more log statements... and the it runs the "dots" coverage tool.. about 700 tests]

................................................................................

................................................................................

................................................................................

................................................................................

................................................................................

................................................................................

................................................................................

................................................................................

................................................................................

..................................DEBUG [web-server]: serving (cached): B:\workspaces\webapps\app1\node_modules\karma\static/context.html

WARN: 'init jasmine.js'

WARN: 'init boot.js'

WARN: 'init adaptor.js'

After the dots reporter runs through, context.html is served up again and then the tests repeat. ( 2x-100x ) 

There are no cases of window.location.reload in the code.

So is some event that is occurring in jasmine 2.0 that causes this test re-run to happen?

I'm not familiar with test runners and I am not sure best how to instrument the code. I have been unable to determine what is causing context.html to be served up again.

Any help would be appreciated! This is a bit frustrating. :)

[Jesse]

Another thing that I have noticed is the jasmineDone callback is not called when the tests repeat but it is called after they all finish.

So if I have 700 tests and they repeat 3 times (2100), the jasmineDone callback occurs only once. 

[Vojta Jína]

Jesse, can you share a simplistic repo (maybe a gist) showing this problem?

I will look into it.

To view this discussion on the web visit https://groups.google.com/d/msgid/karma-users/c6fa0ceb-0abf-4583-9a05-832d0cda33b3%40googlegroups.com.

[Jesse]

I will attempt to do this sometime next week. Thanks for the offer of help.