[Django] #25364: Provide a more generic way to test different browsers in selenium tests
Django
---------------------------------------------+------------------------
Reporter: timgraham | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Keywords:
Triage Stage: Accepted | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------------+------------------------
There is some work in Julien Phalip's
[https://github.com/django/django/compare/master...jphalip:sauce sauce
branch] which allows running the tests against different browsers without
having to creating separate `FooBar<Browser>Tests` classes. It would be
nice to bring this patch current and merge it (we can defer the work in
progress in that branch that allows running the tests on Sauce Labs).
--
Ticket URL: <https://code.djangoproject.com/ticket/25364>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Component: Testing framework | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by akki):
* owner: nobody => akki
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:1>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by akki):
* has_patch: 0 => 1
Comment:
PR: https://github.com/django/django/pull/6099
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:2>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
Left comments for improvement on the pull request.
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:3>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:4>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by akki):
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:5>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: 0 => 1
Comment:
Multiple browsers aren't working on Python 3. Zero tests are run for
`--selenium=firefox,chrome`.
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:6>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Comment (by MoritzS):
I may be a bit late for this, but I see a problem with this approach.
Right now the selenium browser is started in `setUpClass()` and closed in
`tearDownClass()`. This means that the browser has to be started once for
every test class.
This new approach does a bit of Meta class and method copying hacking to
provide a new method `test_foo__browser` for all test functions and
browsers. But then a single test class has tests for ''different''
browsers. This means that the selenium browser started can't be started in
`setUpClass()` so the logic for starting it was moved to `setUp()`. That
however means that the browser is started and closed once for '''every'''
test case. If you consider that starting the browser can take up to a few
seconds (especially on the CI system), I'd say that solution is not
acceptable.
I can think of a solution that works by creating a new test class for each
browser in the meta class' `__new__()` method and then return a proxy
class containing those newly created classes that knows how to run the
tests on all of them.
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:7>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
Yes, I agree that would be better.
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:8>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by MoritzS):
* cc: moritz.sichert@… (added)
Comment:
I ran all selenium tests with firefox and chrome on my 4-core machine and
got the following numbers:
On master branch:
{{{
Ran 67 tests in 124.703s
}}}
On the PR:
{{{
Ran 68 tests in 173.816s
}}}
That's an increase of about 40% even on a powerful desktop machine.
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:9>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by charettes):
* cc: charettes (added)
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:10>
Django
Reporter: timgraham | Owner: akki
Type: New feature | Status: assigned
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:11>
Django
Reporter: timgraham | Owner: akki
Component: Testing framework | Version: master
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette <charette.s@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"44c0ecdd9226d039a8c666b36ae320af2046a1c1" 44c0ecdd]:
{{{
#!CommitTicketReference repository=""
revision="44c0ecdd9226d039a8c666b36ae320af2046a1c1"
Fixed #25364 -- Added generic way to test on all browsers supported by
selenium.
Browser names should be passed as a comma separated list to the --selenium
flag.
Thanks Tim Graham, Simon Charette and Moritz Sichert for review and
discussion.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25364#comment:12>