Android test scenario run aborted randomly (on windows)

[fere...@processmap.com]

Hello everyone!

I currently have a problem: we writing long test scenarios (it might take about 13 minutes for each one) and if i execute one single scenario, it always pass. But if i'm executing all feature file, test run will be aborted after 2-4th scenario.

I run test scenario on local Mac, and all scenarios passed, so we got this problem on windows build agents only.

I got same ruby and gems on windows and mac. Here is gem list:

awesome_print (1.7.0)

bigdecimal (1.2.6)

builder (3.2.2)

calabash-android (0.8.0)

cucumber (2.4.0)

cucumber-core (1.5.0)

cucumber-wire (0.0.1)

diff-lcs (1.2.5)

escape (0.0.4)

gherkin (4.0.0)

httpclient (2.8.2.2)

io-console (0.4.3)

json (1.8.1)

luffa (2.0.0)

minitest (5.4.3)

multi_json (1.12.1)

multi_test (0.1.2)

power_assert (0.2.2)

psych (2.0.8)

rake (10.4.2)

rdoc (4.2.0)

retriable (2.0.2)

rubyzip (1.2.0)

slowhandcuke (0.0.3)

test-unit (3.0.8)

thor (0.19.1)

We are not using Bundler and Gemfiles. We tried to run scenario by "calabash-android run" command and by "rake", and got different logs.

Executing by "calabash-android run" log: calabash_android_run_logs.txt

Executing by rake log: rake_logs.txt

We tried to run this on Ruby 2.2 and Ruby 2.3, and got same result

We also tried to run scenarios not from TeamCity build server, but manually, and got same result, so the reason of problem is not in TeamCity.

[fere...@processmap.com]

I've also noticed, that calabash html report is unfinished (i've attached report.html):

- During first test scenarion, report is created

- after it's failure it adds info about failed and unfinished steps, but this info is not full (Not all tegs are closed).

- when second test scenario is running, it does not make any changes in report

There is no such error, when i run only one test scenario.

[fere...@processmap.com]

[fere...@processmap.com]

Attached report with only one test scenario (only one test has executed @tag)
As You can see, it's finished, not cutted as same test scenario during run for several scenarios (report.html, attached before)

I checked, that it does not depend on steps amount in scenarios. Also I tried to turn off reports at all, but result is same, calabash-android can't finish run after second scenario failure. 

So problem not in report, steps, scenarios and application. Its about how many test scenarios You executing.

I'll appesiate any help or proposal You have, thanks! :>