[Django] #21227: Selenium tests terminate with [Errno 10054]
Django
-------------------------------+----------------------
Reporter: marfire | Owner: nobody
Type: Bug | Status: new
Component: contrib.admin | Version: master
Severity: Normal | Keywords: selenium
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+----------------------
When I run the test suite on Windows with the `--selenium` option, I
consistently see errors like this:
{{{
Exception happened during processing of request from ('127.0.0.1', 51603)
Traceback (most recent call last):
File "C:\Program Other\Python27\Lib\SocketServer.py", line 295, in
_handle_request_noblock
self.process_request(request, client_address)
File "C:\Program Other\Python27\Lib\SocketServer.py", line 321, in
process_request
self.finish_request(request, client_address)
File "C:\Program Other\Python27\Lib\SocketServer.py", line 334, in
finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Django\django\django\core\servers\basehttp.py", line 126, in
__init__
super(WSGIRequestHandler, self).__init__(*args, **kwargs)
File "C:\Program Other\Python27\Lib\SocketServer.py", line 649, in
__init__
self.handle()
File "C:\Program Other\Python27\Lib\wsgiref\simple_server.py", line 116,
in handle
self.raw_requestline = self.rfile.readline()
File "C:\Program Other\Python27\Lib\socket.py", line 447, in readline
data = self._sock.recv(self._rbufsize)
error: [Errno 10054] An existing connection was forcibly closed by the
remote host
}}}
Based on some [http://stackoverflow.com/questions/15244153/whats-the-
proper-way-to-close-a-socket-avoid-errno-10054-with-python-django
anecdotal] [http://stackoverflow.com/questions/13243267/django-and-
selenium-web-testing-error-errno-10054 data], I tried putting a
`refresh()` before the `quit()` in
`contrib.admin.tests.AdminSeleniumWebDriverTestCase._tearDownClassInternal()`
and that reliably solved the problem.
I have no idea why this works, but if it does it's probably worth doing
since it should be harmless.
--
Ticket URL: <https://code.djangoproject.com/ticket/21227>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timo):
* needs_better_patch: => 0
* stage: Unreviewed => Accepted
* component: contrib.admin => Testing framework
* needs_tests: => 0
* needs_docs: => 0
Comment:
There are some other places where we call `selenium.quit()` -- should
those places be updated as well?
docs/topics/testing/overview.txt: cls.selenium.quit()
tests/view_tests/tests/test_i18n.py: cls.selenium.quit()
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:1>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by marfire):
I think it's worth doing this wherever we're calling `selenium.quit()`,
since it seems harmless and does solve the spurious failures. Here's a
patch for it: https://github.com/django/django/pull/1806.
However, I hesitate to give this workaround canonical status by putting it
in the documentation, since it's basically a bug in another project, could
be fixed at any time, and adds a little extra noise and cognitive
dissonance to the documentation. It only seems to pop up in some
environments, and the workaround can be easily found with a quick internet
search. On the other hand, for people developing reusable applications, it
might be nice to warn them about this since it might affect them even if
they aren't seeing the bug in their own test environments. Maybe the wiki
is the right place for this?
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:2>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timo):
* status: new => closed
* resolution: => fixed
Comment:
In 08c9ab5a0f564a3ac7803e6a97fae855f2e0524e:
Fixed #21227 -- Added workaround for selenium test failures
Added a refresh() before quit() in the selenium tests, since this
solves the problem of spurious test failures in some environments.
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:3>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Severity: Normal | Resolution: fixed
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by apollo13):
@marfire That workaround slows down firefox pretty much and should we
unneeded; can you test if this sequence works better for you:
{{{
if hasattr(cls, 'selenium'):
cls.selenium.close()
super(AdminSeleniumWebDriverTestCase,
cls)._tearDownClassInternal()
if hasattr(cls, 'selenium'):
cls.selenium.quit()
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:4>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Severity: Normal | Resolution: fixed
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by apollo13):
@marfire: Is this a hard failure or just output on the console? Cause
essentially everything which is a subclass of Exception is captured and
more or less ignored.
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:5>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Severity: Normal | Resolution: fixed
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by apollo13):
* cc: apollo13 (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:6>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Severity: Normal | Resolution: fixed
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by marfire):
I'm out of town right now, but I'll try this next week and report back...
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:7>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Severity: Normal | Resolution: fixed
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Florian Apolloner <florian@…>):
In [changeset:"7d0a5190328f277b97b1f55171e1fecb22a1e461"]:
{{{
#!CommitTicketReference repository=""
revision="7d0a5190328f277b97b1f55171e1fecb22a1e461"
Revert "Fixed #21227 -- Added workaround for selenium test failures"
This reverts commit 08c9ab5a0f564a3ac7803e6a97fae855f2e0524e.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:8>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by apollo13):
* status: closed => new
* resolution: fixed =>
Comment:
I just reverted it since it doubles (or more) the time for firefox tests.
The real fix should go something like this: When we tear down set a flag
that we are in a shutdown process and then ignore the error.
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:9>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by tkhyn):
Hi,
An attempt to mitigate this issue is in attached patch above.
The only way to make it work is to Attempt to set
server_thread.httpd.ignore_errors = True before calling WebDriver.quit()
in the Selenium test case class's tearDownClass, like that:
{{{
class MySeleniumTestCase(LiveServerTestCase)
[...]
@classmethod
def tearDownClass(cls):
if hasattr(cls, 'server_thread'):
# test if server_thread attribute is available (as there may
have been an exception in setUpClass)
# setting ignore_errors flag on WSGI server thread to avoid
unwanted 10054
cls.server_thread.httpd.ignore_errors = True
cls.wd.quit() # cls.wd is the WebDriver instance
super(LiveServerTestCase, cls).tearDownClass()
}}}
Could not find a simpler way to do it. Fully integrating it in Django
would require a django-specific SeleniumTestCase class on top of
LiveServerTestCase
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:10>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timo):
#22252 was a duplicate.
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:11>
Django
Reporter: marfire | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by anonymous):
For anyone looking to repro this, here's a single test case that shows the
error:
python runtests.py --selenium
admin_widgets.tests.DateTimePickerSeleniumFirefoxTests
be aware that if this PR on logging gets accepted
(https://github.com/jmbowman/django/commit/c34c5860d4e842e62574657ec61a125bc91b6c0f)
then the error will no longer be visible on stdout. If I get time I'll
try and build some kind of specific failing test for just this issue...
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:12>
Django
Reporter: Kevin Christopher | Owner: nobody
Henry |
Type: Bug | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tyeth Gundry):
This has recently bitten me when testing locally.
Following theoldtestinggoat.com TDD in python tutorial, and decided to
switch to chrome instead of firefox, and the tests pass but with this
error logged each time when in conjunction with a local django test
server, if tested against a remote host the connection gets closed cleanly
and no error is reported.
Adding the driver.refresh() call before close() alleviated my chrome-based
woes, that and disabling gpu acceleration when calling chrome in headless
mode.
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:13>
Django
Reporter: Kevin Christopher | Owner: nobody
Henry |
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by mrts):
I got rid of the `ConnectionResetError` by creating
`ConnectionResetErrorSwallowingLiveServerThread` as follows that swallows
the error:
{{{
import socket
import errno
from django.core.servers.basehttp import ThreadedWSGIServer
from django.test.testcases import QuietWSGIRequestHandler,
LiveServerThread
class
ConnectionResetErrorSwallowingQuietWSGIRequestHandler(QuietWSGIRequestHandler):
def handle_one_request(self):
try:
super().handle_one_request()
except socket.error as err:
if err.errno != errno.WSAECONNRESET:
raise
class ConnectionResetErrorSwallowingLiveServerThread(LiveServerThread):
def _create_server(self):
return ThreadedWSGIServer((self.host, self.port),
ConnectionResetErrorSwallowingQuietWSGIRequestHandler,
allow_reuse_address=False)
}}}
It can be used it in tests that extend `StaticLiveServerTestCase` as
follows:
{{{
class AdminSiteTestCase(StaticLiveServerTestCase):
server_thread_class = ConnectionResetErrorSwallowingLiveServerThread
}}}
@timgraham, what do you think, could something like this be added to
`django.test.testcases`?
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:14>
Django
Reporter: Kevin Christopher | Owner: nobody
Henry |
Component: Testing framework | Version: master
Keywords: selenium | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* status: new => closed
* resolution: => fixed
Comment:
Fixed in 772eca0b0219f63129282c3596fc210951433c13 because `WSGIServer` now
suppress `ConnectionResetError`.
--
Ticket URL: <https://code.djangoproject.com/ticket/21227#comment:15>