[Django] #28968: LiveServerTestCase prematurely closing HttpResponse with Django 2.0
Django
---------------------------------------------+------------------------
Reporter: Alexander Todorov | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 2.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------------+------------------------
I'm trying to migrate an existing project to Django 2.0 and I'm hitting a
problem with some of our tests, all of the problematic tests are very
similar in nature:
1) They are all LiveServerTestCase based
2) All of them have a XML-RPC client which issues requests to the Django
app
3) Using django-modern-rpc the application under test is able to execute a
function and return a result
3) All error out on the first request (most of the times) with traceback
from the xmlrpc client module in Python telling me that that connection
has been closed without a response.
Here are the few data points I was able to collect:
* Locally happens on both MySQL and SQLite but on SQLite less frequently,
looks like a race condition. In Travis CI happens on SQLite but not on
MySQL so go figure!
* with a local MySQL instance I'm able to reproduce all the times
* the root-cause seems to be in wsgiref.handlers.py::BaseHandler.run()
which calls finish_response() which executes HttpResponse.close(). In
run() there's also a comment about async servers and that they should not
call .close() inside .finish_response()
* if I modify Django's basehttp.py::ServerHandler.http_version to '1.0'
all tests seem to pass regardless if we use a multi-threaded or single-
threaded LiveServer!
* This is on Python 3.5, RHEL 7 system.
So it looks like the problem is with ServerHandler (presumably tests)
wanting to use HTTP 1.1 but I can't narrow it down firther ATM.
To reproduce you may checkout the code at
https://github.com/kiwitcms/Kiwi;
pip install -r requirements/devel.txt; pip install --upgrade Django and
./manage.py test --noinput --settings=tcms.settings.test.mysql
tcms.xmlrpc.tests.test_logging.TestXMLRPCLogging.test_logging_with_authenticated_user
--
Ticket URL: <https://code.djangoproject.com/ticket/28968>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Alexander Todorov | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* cc: Tom Forbes (added)
Comment:
It sounds similar to ac756f16c5bbbe544ad82a8f3ab2eac6cccdb62e. I'm not
sure if anyone will be interested in debugging your project to find if
Django is at fault.
--
Ticket URL: <https://code.djangoproject.com/ticket/28968#comment:1>
Django
Reporter: Alexander Todorov | Owner: nobody
Component: HTTP handling | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* status: new => closed
* resolution: => needsinfo
Comment:
Please reopen if you can explain why Django is at fault.
--
Ticket URL: <https://code.djangoproject.com/ticket/28968#comment:2>
Django
Reporter: Alexander Todorov | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Alexander Todorov):
* status: closed => new
* resolution: needsinfo =>
Comment:
@Tom Forbes,
thanks for the hint, I've also been looking around the sme parts of the
code. Reverting the commit you listed above and testing with
ServerHandler.http_version = '1.1' makes all of my tests to PASS but
'./manage.py test' doesn't exit!.
All of my responses also include the Content-Length header. I will let you
know when I find more.
--
Ticket URL: <https://code.djangoproject.com/ticket/28968#comment:3>
Django
Reporter: Alexander Todorov | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Alexander Todorov):
I think there's a bug inside Python's xmlrpc.client.Transport class which
isn't playing nicely when Django closes the connection when HTTP 1.1 is in
use. In particular this piece of code in xmlrpc/client.py
{{{
1129 def request(self, host, handler, request_body, verbose=False):
1130 #retry request once if cached connection has gone cold
1131 for i in (0, 1):
1132 try:
1133 return self.single_request(host, handler,
request_body, verbose)
1134 except OSError as e:
1135 if i or e.errno not in (errno.ECONNRESET,
errno.ECONNABORTED,
1136 errno.EPIPE):
1137 raise
1138 except http.client.RemoteDisconnected:
1139 if i:
1140 raise
}}}
if I swap the except blocks then everything seems to work for me
(ac756f16c5bbbe544ad82a8f3ab2eac6cccdb62e is applied).
About the statement above that './manage.py test' doesn't terminate: I
have the feeling that the server is still running while there's no client
to issue more requests (test has finished). I'm not sure if Django can do
anything about this.
I'd love to hear your comments on my findings since I don't know this area
of Django or Python's http/xmlrpc libraries very well but feel free to
close.
--
Ticket URL: <https://code.djangoproject.com/ticket/28968#comment:4>
Django
Reporter: Alexander Todorov | Owner: nobody
Component: HTTP handling | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* status: new => closed
* resolution: => invalid
Comment:
I don't have any insight to offer.
--
Ticket URL: <https://code.djangoproject.com/ticket/28968#comment:5>
Django
Reporter: Alexander Todorov | Owner: nobody
Component: HTTP handling | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tom Forbes):
I don't have much to add either, except that it would be good to open a
ticket with Python if it is indeed an issue in xmlrpc. If this is causing
issues with a core standard library, which is hopefully we'll tested, then
this could be triggering other libraries and code that is not reported.
I have some time this weekend to review this ticket and perhaps dig into
it a bit, but it would be nice to just be able to enable keep alive in all
but streaming requests.
--
Ticket URL: <https://code.djangoproject.com/ticket/28968#comment:6>
Django
Reporter: Alexander Todorov | Owner: nobody
Component: HTTP handling | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Alexander Todorov):
FYI the Python bug is:
https://bugs.python.org/issue26402
--
Ticket URL: <https://code.djangoproject.com/ticket/28968#comment:7>