{{{
$ ./runtests.py servers.tests
}}}
This is with Python 3.5.2 on Mac OSX.
There is no error message. It just hangs. It works okay when passing
`--parallel 1`.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
I can't reproduce on Ubuntu 14.04, Python 3.5.2. Just in case, how many
parallel process do you have? I have 4 here.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:1>
Comment (by cjerdonek):
8, but it also happens when I pass `--parallel 2`. To narrow it down, it
happens when I run the following (I believe the test setup executes, but
it never gets to running the tests):
{{{
$ ./runtests.py servers.tests.LiveServerDatabase
servers.tests.LiveServerViews
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:2>
Comment (by timgraham):
You could check if the issue is present at the time that parallel test
running was added in Django and if not, bisect to find the commit that
introduced the problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:3>
Comment (by bpeschier):
Just ran the tests at the mentioned commit on my macOS Sierra public beta
with a fresh 3.5.2 python environment. No problems there.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:4>
* status: new => closed
* resolution: => worksforme
Comment:
Feel free to reopen if you find that Django is at fault.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:5>
Comment (by cjerdonek):
I narrowed this down further to the following:
The hang happens when calling `urllib.request.urlopen()` inside
`LiveServerBase`. And digging further, inside CPython's request.py, the
hang happens when calling `_scproxy._get_proxies()`.
There seem to be known issues around `_scproxy` in CPython for Mac OS X
users. For example, see the following two issues in CPython's bug
tracker:
(1) Apple-supplied libsqlite3 on OS X is not fork safe; can cause
crashes:
http://bugs.python.org/issue27126
(See this comment, in particular:
http://bugs.python.org/issue27126#msg266405 )
(2) exception error in _scproxy.so when called after fork
http://bugs.python.org/issue13829
There is a chance that there are easy workarounds on the Django side that
can be made to accommodate Mac OS X users.
Also, for the record I'm using 10.11.6.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:6>
* status: closed => new
* resolution: worksforme =>
Comment:
I was able to get things working by causing Mac OS X's auto-detection of
proxy information to be skipped. This possibility is mentioned
[https://docs.python.org/3/library/urllib.request.html#urllib.request.ProxyHandler
here] in CPython's `urllib.request.ProxyHandler` documentation.
More detailed instructions appear in `urllib.request`'s module docstring.
To disable the auto-detection, I followed the instructions in the above-
mentioned module docstring and modified `LiveServerBase.urlopen()` to look
like the following (this is just a proof of concept):
{{{#!python
def urlopen(self, url):
import urllib.request
# Pass an empty dictionary to disable auto-detecting the proxy.
proxy_support = urllib.request.ProxyHandler({})
opener = urllib.request.build_opener(proxy_support)
urllib.request.install_opener(opener)
return urlopen(self.live_server_url + url)
}}}
After doing this, the following command worked fine:
{{{
$ ./runtests.py servers.tests
}}}
Since this seems like a fundamental bug / limitation on Mac OS X (in at
least some versions), it seems worth considering including Mac-specific
code to enable the parallel running of tests on this platform.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:7>
* component: Uncategorized => Core (Other)
* type: Uncategorized => Cleanup/optimization
* stage: Unreviewed => Accepted
Comment:
I wonder what's special about your system compared to other Mac OS
contributors who didn't encounter the problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:8>
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:9>
Comment (by cjerdonek):
I've been unable to post my full comment because of the spam blocker, so
I'll try including only the link I wanted to mention (but broken apart to
evade the blocker).
The first part of the link is here: http://www.evanjones.ca/
The second part is here: fork-is-dangerous.html
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:10>
Comment (by cjerdonek):
Also (to add more of what I wanted to say but couldn't above), I don't
know yet why I'm the first person to notice / report this. In case it
would help, I started working on a reduced way to reproduce this
(independent of the Django test suite).
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:11>
Comment (by cjerdonek):
Oh, the person that wrote the blog post already created a "reduced test
case" that I started to look for: https://github.com/evanj/fork-thread-
bugs/blob/master/osx_python_crash_v1.py
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:12>
* has_patch: 0 => 1
Comment:
Here is a documentation improvement showing a workaround:
https://github.com/django/django/pull/11203
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:13>
Comment (by Tim Bell):
I have encountered the same problem (on MacOS 10.14.4 Mojave) and confirm
that the solution works for me. This would be a useful addition to the
documentation.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:14>
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:15>
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:16>
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:17>
* status: new => closed
* resolution: => wontfix
Comment:
Closing as per discussion on PR: lack of a decent reproduce, plenty of
Google results for the error, not our responsibility, and likely a problem
that's been resolved in recent releases (but c.f. lack of reproduce...)
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:18>
* status: closed => new
* resolution: wontfix =>
Comment:
I'm at the sprints at DjangoCon. ''Many'' new contributors have hit this,
so I think the ''doesn't reproduce'' worry is passed. I'm going to re-open
this accordingly. We should doc so new contributors are able to get going.
Related is #30806 (which is targeting running `tox` specifically.)
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:19>
* owner: nobody => Carlton Gibson
* status: new => assigned
* stage: Ready for checkin => Accepted
Comment:
I just want to adjust this, so moving to Accepted and assigning to myself.
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:20>
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"5d9cf79baf07fc4aed7ad1b06990532a65378155" 5d9cf79b]:
{{{
#!CommitTicketReference repository=""
revision="5d9cf79baf07fc4aed7ad1b06990532a65378155"
Fixed #27086 -- Doc'd fix for test suite hang on macOS.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:21>
Comment (by Carlton Gibson <carlton.gibson@…>):
In [changeset:"fffce1ac4c412ec49bde659b79dd943266c9434f" fffce1a]:
{{{
#!CommitTicketReference repository=""
revision="fffce1ac4c412ec49bde659b79dd943266c9434f"
[3.0.x] Fixed #27086 -- Doc'd fix for test suite hang on macOS.
Backport of 5d9cf79baf07fc4aed7ad1b06990532a65378155 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:22>
Comment (by Carlton Gibson <carlton.gibson@…>):
In [changeset:"4f7ba25e67681efbb67cc4810e919aa7369cbe34" 4f7ba25e]:
{{{
#!CommitTicketReference repository=""
revision="4f7ba25e67681efbb67cc4810e919aa7369cbe34"
[2.2.x] Fixed #27086 -- Doc'd fix for test suite hang on macOS.
Backport of 5d9cf79baf07fc4aed7ad1b06990532a65378155 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27086#comment:23>