problems aggregating reports with too many tests (java.nio.file.FileSystemException: too many open )

[Dmitriy Grablyov]

Has anyone faced problems with aggregating too many tests into one report?

Now we have (as serenity reports) almost 1000 Jbehave scenarios with 2160 test step executed.

So tests are running in 12 physical threads at different time and than we need to aggregate them into a general report.

Recently we have noticed that this becomes a problem for serenity (or jenkins, I still do not understand).

At some point aggregator starts logging exceptions like:

java.nio.file.FileSystemException: /var/lib/jenkins/jobs/automation_QA_AUTO_1.01.2_serenity-BDD_aggregate_reports/workspace/frontier_autotests/all-tests-parent/serenity-bdd-tests/target/site/serenity/browser-iexplorer.properties: Too many open files

performing 

lsof | grep -i  /var/lib/jenkins/jobs/automation_QA_AUTO_1.01.2_serenity-BDD_aggregate_reports | wc -l

on server gives me a result of 50000+ open file descriptors for the same file.

We are still using 1.1.5 , I did not try this on the latest version yet since we still need to perform investigation about whether new version is suitable for our tests. Latest serenity version appears to be much slower at the reports aggregation phase also.

Any suggestions and solutions are appreciated.

If someone faced this problem and is sure that this is fixed in later versions of serenity - that would be great.

Best Regards,

Dmytro Grablov

[Erik]

We are seeing the same issues when running the build using Bamboo and with Cucumber instead of JBehave.

The only fix we could find for now is take no screenshots except on errors:

serenity.properties

serenity.take.screenshots = FOR_FAILURES

It's quite a problem since we don't have control in our environment over the configuration of the platform that we're doing the nightly builds on....

[John Ferguson SmartSmart]

What version of Serenity are you using? The latest version no longer writes screenshots to the filesystem during processing to avoid having too many open file descriptions.

Dmitry: When you say "on server gives me a result of 50000+ open file descriptors for the same file”, do you know what file that is?

___________________________________________________
John Smart | Wakaleo Consulting  |  +44 7497151337
Making smart teams collaborate better
http://www.wakaleo.com  |  john....@wakaleo.com
___________________________________________________

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-use...@googlegroups.com.
To post to this group, send email to thucydid...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dmitriy Grablyov]

Hi John,

sorry for the delay.

From that point we tried to investigate this issue again.

Now we are using serenity 1.1.5 and the error from logs referenced to a file browser-chrome.properties

I tried the latest release 1.1.26 and this error does not appear anymore.

Nevertheless what we are facing right now (for both serenity versions) is the excessively large report size and the excessively large aggregation time.

Now we have about 1000 stories which result in about 1500 test scenarios and about 2000 tests because of test data parameterization.

So in this case full report aggregation takes 60+ minutes in time scope and final result has a size of about 1.5 GB

Is there any possibility to reduce the size of the report? we take screenshots only on failures, but set the reporting to track every step execution. We could also disable this feature though for the failed scenarios we still need a full flow of steps execution.

Because what I see in the final report is the large files. Index.html itself weighs about 5 MB, and there are many files with size > than 500 KB. Mostly these are requirements representations.

Best Regards,

Dmitriy Grablyov.