Good ways to slow down a test run?

[Nick Shaw]

Hey everyone, this is a repost from a SO question but answers there were more Selenium specific than using Robot Framework directly, do you guys have any responses to add?

I'm reasonably new to Robot Framework and have run into various forms of this issue a number of times now. Essentially I create a suite of tests which eventually pass and check for the things I expect, they then go through a CI/CD pipeline and on the dedicated test runners I hit some failures. In essence this seems to be because the test runners do things a little faster, whether due to hardware or selenium speed, it doesn't really matter.

To avoid these failures I end up adding to the tests to 'slow' them down at certain points to ensure success, basically bloating them or just being a little more protective. So for those with more experience, how do you handle this situation? I'd like to stick with best practice as far as I can, i.e. I'm not just using sleeps everywhere but using implicit waits, still I don't know what the best approaches are.

Here's snippet of an issue I've hit recently:

Wait Until Page Contains Element ${STATIC_TABLE_XPATH}/somepath 10
${image_text} = Selenium2Library.Get Element Attribute ${STATIC_TABLE_XPATH}/somepath
${second_paragraph_text} = Selenium2Library.Get Text ${STATIC_TABLE_XPATH}/somepath

So basically despite these being functionally correct, they pull back nothing sometimes when run on the test runners. I figure this is because things are happening too quickly, so I added a Wait Until Page Contains Element, unfortunately the same problem still exists. Is there a best way to handle this situation so it consistently passes?

I thought adding a wait until keyword succeeds on the expected variable value would be pretty horrible but happy to be proved wrong!

Thank you

[Mikko Korpela]

I think that the problem must be somewhere else.

Are you sure that the system your testing is not buggy? Are you sure that all the actions in your test before this have happened correctly?

Best practice is to have guard actions (for example Wait Until..) that check that the system under test is in the right state and not have "blind" sleeps that just assume that the system under test is in the right state after the sleep.

Cheers,

Mikko

[Nick Shaw]

Would you have any advice for figuring out the root problem?

To be honest the system application under test doesn't appear buggy to me but there's definitely strange stuff happening.  i.e. I run my tests locally, they pass, I run them manually on a shared environment, they pass, I run them while remoting into the test runner, they pass, however if I leave them running on the test runner and don't remote in, I get 1 or 2 flakey failures.  It's a tad frustrating!

[Mikko Korpela]

You could try to totally stop the test execution when the problem happens and manually later check what is the situation.

-- 

Mikko Korpela

--
You received this message because you are subscribed to a topic in the Google Groups "robotframework-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/robotframework-users/XAMK4JcN70A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

[M Hanes]

Have your tried using Set Selenium Speed? I used Set Selenium Speed .1 seconds and it slowed the test down just a bit.

[Tatu Aalto]

Ugh

And if you execute 1000 S2L keywords in a test you slow down the test by 1:30. And if you have 60 test, your execution time did just grow by hour and half.

But the original problem is quite typical in web applications. Also it is one which doesn't not have a single correct answer. For me, it is an indication of a timing problem, in somewhere (in RF data or application side, usually but definitely not limited to these). I usually discuss with the developer, that I did encountered this problem, when I do these steps and I use these as selectors. Usually, in my application, I am polling wrong thing to wait something to complete. Also few times it has been a problem in the application side and we have made a fix for it. Also, in very rare cases, I have used small (< 0.5s) sleep to avoid some JavaScript issues. There are also numerous other things which could (and will) go wrong.

False positives and especially flashing tests are pain in... But it's the nature of the web applications and it is a constant battle against, which you or I won't ever win. But I am doing it, in my company and I am loving my job. There are plenty of challenges, but that is what makes it fun.

-Tatu
Send from my mobile

On 9 Sep 2015 2:37 am, "M Hanes" <mc41...@gmail.com> wrote:

Have your tried using Set Selenium Speed? I used Set Selenium Speed .1 seconds and it slowed the test down just a bit.

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.