Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
[signin] stabilizing selenium tests
3 views
Skip to first unread message
Shane Tomlinson
May 14, 2013, 1:43:28 PM5/14/13
to dev-id...@lists.mozilla.org
I had some extra time in Vancouver, so I decided to get our Selenium
tests back in order.
We were seeing a lot of random timeouts and errors, and intermittently
failing tests makes me lose faith in the tests in general.
After about 100 iterations of the full suite, two consistent problems
were found: eyedee.me is slow and one test calls "done" twice.
I replaced all calls to eyedee.me with calls to testidp.org (this was
suggested by Lloyd a while back). Next, I fixed the test that called its
"done" function twice.
When testing against a c1.medium ephemeral instance using 5 parallel
tests, I have seen 25 consecutive iterations of the full suite run
without a single failure in both Firefox and Chrome.
There are still occasional failures, but they are significantly reduced.
The PR is ready for review [1].
Shane
===========
[1] - https://github.com/mozilla/browserid/pull/3388
tests back in order.
We were seeing a lot of random timeouts and errors, and intermittently
failing tests makes me lose faith in the tests in general.
After about 100 iterations of the full suite, two consistent problems
were found: eyedee.me is slow and one test calls "done" twice.
I replaced all calls to eyedee.me with calls to testidp.org (this was
suggested by Lloyd a while back). Next, I fixed the test that called its
"done" function twice.
When testing against a c1.medium ephemeral instance using 5 parallel
tests, I have seen 25 consecutive iterations of the full suite run
without a single failure in both Firefox and Chrome.
There are still occasional failures, but they are significantly reduced.
The PR is ready for review [1].
Shane
===========
[1] - https://github.com/mozilla/browserid/pull/3388
0 new messages