LiveServerTestCase non-deterministic Remote end closed connection without response

[Rares Vernica]

Hello,

I have a test script starting a live server using LiveServerTestCase. The test script issues requests to the server using requests.Session. At random points in the script the Django server decides to close the connection without a response. Here are some more details.

My test script is not testing a Django application, but a non-Django component that is supposed to interact with a Django Rest Framework. So, my non-Django component uses requests.Session to connect and interact with Django. Hence my test script uses requests.Session as well.

This works well on Python 2.7 and Django 1.11.15, but I'm migrating to Python 3.6 and Django 2.1.1. I see my test script making progress through the test case, and at a random point it triggers the following exception:

----------------------------------------------------------------------
Traceback (most recent call last):
  File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib64/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "...local/lib/python3.6/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "...local/lib/python3.6/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "...local/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib64/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

...
  File "...local/lib/python3.6/site-packages/requests/sessions.py", line 559, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "...local/lib/python3.6/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "...local/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "...local/lib/python3.6/site-packages/requests/adapters.py", line 495, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

Repeated runs of the test script will progress more or less through the test cases. It is not deterministic.

On start-up the test script creates a requests.Session and authenticates to Django. Then it will issue GET/POST requests against a Django Rest Framework API. I wonder why are the connection closed. I'm using requests 2.19.1.

Does this sound familiar? Is this related to a change in Django 2 that I'm missing?

Thanks!

Rares

[amir...@gaa9.com]

Any updates on this issue? I'm having the same issue right now.

[AdrianB]

I'm also seeing this with a slightly simpler scenario that also involves a requests.get inside a LiveServerTestCase. Success/failure is random with failures reporting the same ConnectionError message. Any ideas?