Serenity tests running more slowly as more test are added

[mirosla...@gmail.com]

Hi guys,

I am using:

net.serenity-bdd:serenity-core:1.1.42
net.serenity-bdd:serenity-gradle-plugin:1.1.42
net.serenity-bdd:serenity-cucumber:1.1.11

net.serenity-bdd:serenity-rest-assured:1.1.42
net.serenity-bdd:serenity-report-resources:1.1.41

Versions are the best combination, when I am not problem with report formating or other issues for running serenity.

The problem is, that we have about 700 tests now (backend REST testing) and we are experiencing strange behavior. As test are running there is increase of time between the tests, when nothing is happening (no log in console, no interaction with db, ...).

Can it be, that serenity is doing something like caching information for the report, and as this is more and more bigger as more test do run, it takes more time to do it? When the test run, even the last ones, there is no problem, each test run fast, but there is still bigger delay between tests as more test already run. 

This is causing, that on Jenkins job takes about one and half hour, but in report I see 15 minutes :-D.

Or is there a setting that can help?

Thanks for the help.

[SeleniumWhat?]

I guess, it is likely to be the Serenity Report behavior.  it requires a large chunk of memory for it to store the aggregated report images while executing.

  As you run a large number of test cases you would run into this issue of memory drainage.  Turn off the Serenity Report feature and see if you see the difference.

[John Smart]

Try using the latest version of Serenity (1.2.2)

On 17 January 2017 at 15:11, <mirosla...@gmail.com> wrote:

Hi,

thanks for the tip... but unfortunately I am not able to find how to switch it off, can You please help me?

Dňa sobota, 14. januára 2017 1:27:39 UTC+1 SeleniumWhat? napísal(-a):

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

___________________________________________________
John Smart | Wakaleo Consulting  |  +44 7398 832273
Making smart teams collaborate better
http://johnfergusonsmart.com  |  john....@wakaleo.com
___________________________________________________

We love breaking down silos and helping smart teams collaborate better! Ask about our tailored on-site workshops in Agile Product Planning, BDD Requirements Discovery,  BDD, TDD and Clean Coding, and Advanced BDD Test Automation.

___________________________________________________

[Miroslav]

Hi,

thanks for the tip... but unfortunately I am not able to find how to switch it off, can You please help me?

Dňa sobota, 14. januára 2017 1:27:39 UTC+1 SeleniumWhat? napísal(-a):

I guess, it is likely to be the Serenity Report behavior.  it requires a large chunk of memory for it to store the aggregated report images while executing.

  As you run a large number of test cases you would run into this issue of memory drainage.  Turn off the Serenity Report feature and see if you see the difference.

[Miroslav]

Thanks,

will try that, hopefully it does not mess with the report...

Dňa utorok, 17. januára 2017 16:12:49 UTC+1 John Smart napísal(-a):

Try using the latest version of Serenity (1.2.2)

On 17 January 2017 at 15:11,  wrote:

Hi,

thanks for the tip... but unfortunately I am not able to find how to switch it off, can You please help me?

Dňa sobota, 14. januára 2017 1:27:39 UTC+1 SeleniumWhat? napísal(-a):

I guess, it is likely to be the Serenity Report behavior.  it requires a large chunk of memory for it to store the aggregated report images while executing.

  As you run a large number of test cases you would run into this issue of memory drainage.  Turn off the Serenity Report feature and see if you see the difference.

On Friday, January 13, 2017 at 9:43:39 AM UTC-5, mirosla...@gmail.com wrote:

Hi guys,

I am using:

net.serenity-bdd:serenity-core:1.1.42
net.serenity-bdd:serenity-gradle-plugin:1.1.42
net.serenity-bdd:serenity-cucumber:1.1.11

net.serenity-bdd:serenity-rest-assured:1.1.42
net.serenity-bdd:serenity-report-resources:1.1.41

Versions are the best combination, when I am not problem with report formating or other issues for running serenity.

The problem is, that we have about 700 tests now (backend REST testing) and we are experiencing strange behavior. As test are running there is increase of time between the tests, when nothing is happening (no log in console, no interaction with db, ...).

Can it be, that serenity is doing something like caching information for the report, and as this is more and more bigger as more test do run, it takes more time to do it? When the test run, even the last ones, there is no problem, each test run fast, but there is still bigger delay between tests as more test already run. 

This is causing, that on Jenkins job takes about one and half hour, but in report I see 15 minutes :-D.

Or is there a setting that can help?

Thanks for the help.

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-use...@googlegroups.com.
To post to this group, send email to thucydid...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Miroslav]

Hi John,

I tried the higher version. Report is generated, but I found that:

- execution is still long, no improvement at all

- and Serenity is not recognizing @skip tag, and also any of my custom @tags, which I was able to see also in report before > Do I need to set some property to make it work again?

Thanks.

Dňa utorok, 17. januára 2017 16:12:49 UTC+1 John Smart napísal(-a):

Try using the latest version of Serenity (1.2.2)

[John Smart]

Could you share your project?

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Miroslav]

Hi John,

I would try...

Meanwhile I did some testing, and I found out that tags are related to serenity-cucmber plugin version: ( serenity-core and others are v1.2.2 )

v1.23 > is ignoring any tags expect @pending (not showing any tags on dashboard)

v1.20 > is ignoring any tags expect @pending and @skip (not showing any tags on dashboard)

v1.18 > same as 1.20

v1.16 > is working properly and is showing all custom tags on dashboard, and also showing @issue tags for links to JIRA

Are the changes intentional or it is a bug?

Dňa streda, 18. januára 2017 10:14:15 UTC+1 John Smart napísal(-a):

Could you share your project?

On 18 January 2017 at 08:53, Miroslav <mirosla...@gmail.com> wrote:

Hi John,

I tried the higher version. Report is generated, but I found that:

- execution is still long, no improvement at all

- and Serenity is not recognizing @skip tag, and also any of my custom @tags, which I was able to see also in report before > Do I need to set some property to make it work again?

Thanks.

Dňa utorok, 17. januára 2017 16:12:49 UTC+1 John Smart napísal(-a):

Try using the latest version of Serenity (1.2.2)

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-use...@googlegroups.com.
To post to this group, send email to thucydid...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.