Tests repeat for karma + mocha + angular4

[Arran Bartish]

I've converted a basic project generated by angular-cli 1.0.0 from using jasmine to using mocha. For reasons I do not understand when I convert to use mocha, all the tests run twice. Interestingly when we run the same tests/code with wallabyjs runner, or with a downgraded angular2, then the tests only run once.

To reproduce from scratch I took 3 steps with an optional 4th. Unfortunately there is a lot of code involved, so I've provided a repository to demonstrate and a command to follow along each step which can be found in the step detail.

1. create project with angular-cli 

$ ng new someBlankProject
$ cd someBlankProject
$ npm run test  -- --single-run=true

At this stage I expect 3 tests run and we "pass"

Detail

setup: angular-cli + angular4 + karma + jasmine

command to reproduce 

$ git clone git@github.com:arranbartish/angular-cli-seed.git fresh-project; cd fresh-project; git checkout 2bf12b577173085344a86e37726d9f7ece930c77;npm install;npm run test -- --single-run=true

2. validate that jasmine tests run using karma and wallaby both locally and on CI (travis)

Update the basic project to include Travis CI and the wallabyjs runner

$ npm run test  -- --single-run=true

At this stage I expect 3 tests run and we "pass"

Confirmed against wallabyjs 

No failing tests, 3 passing

Detail

setup: angular-cli + angular4 + karma + jasmine

command to reproduce

$ git clone git@github.com:arranbartish/angular-cli-seed.git add-wallaby; cd add-wallaby; git checkout 0bb96884d0e71f286d0b4fedc0dcafd20dc9d2b1;npm install;npm run test -- --single-run=true

3. introduce mocha and convert specs from jasmine to mocha

Once al lthe specs are converted and karma has been reconfigured to use mocha

npm run test  -- --single-run=true

At this stage we again expect 3 tests run but we "fail" with actual result 6 tests run

    - `Chromium 53.0.2785 (Ubuntu 0.0.0): Executed 6 of 3 SUCCESS (0.347 secs / 0.042 secs)`

Confirmed against wallabyjs 

No failing tests, 3 passing

Detail

setup: angular-cli + angular4 + karma + mocha   |

command to reproduce 

$ git clone git@github.com:arranbartish/angular-cli-seed.git converted-mocha; cd converted-mocha; git checkout 0a9ed8804e15c451ff0d67ebd2d38980d54f9763;npm install;npm run test -- --single-run=true

Observation 

One of the strangest things that I've observed is that using angular 2 does not have the same behavior! 

4. downgrade angular from 4 to 2

Just update the package.json and use angular 2.4 instead of 4.

npm run test  -- --single-run=true

If the problem is the mocha+karma setup, then I would expect to still have 6 of 3 in the result. Instead we have a "pass" with only 3 tests run

Confirm against wallabyjs 

No failing tests, 3 passing

Detail

setup: angular-cli + angular2 + karma + mocha

command to reproduce

git clone git@github.com:arranbartish/angular-cli-seed.git downgrade-angular; cd downgrade-angular; git checkout 0e6c2811b9aab722fcc382e4676d97c089ad1f91;npm install;npm run test -- --single-run=true

I've setup a commit chain here on GitHub (commits referenced in the above commands) with each step described above. You can see each diff and the Travis build which shows the weird repeating tests at step 3. 

I'm very confused as to where to go from here. I suspect it's karma related since it seems that the wallabyjs runner stay consistent throughout. Any more advice would be massively helpful. 

Thanks,

Arran

[Arran Bartish]

Just in case some finds this in the future.

There is more detail in a resolved karma issue I raised and on Stack Overflow. Spoiler: It turns out to be user (my) error where I had defined too many frameworks in my frameworks array. (Face palm)