[Django] #30361: Watchman timing out when loading server
Django
-----------------------------------------+------------------------
Reporter: Jacobkg | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
When starting the server on my project using `./manage.py runserver`, I
get the error message:
`Error connecting to Watchman: timed out waiting for response, while
executing ('watch-project', 'my_app')`
I am using Watchman v4.9 on an Ubuntu 16.04 Docker image hosted on Mac OS
X 10.14.4.
My research shows that there is a default 1 second timeout in `pywatchman`
that Django is using. In my tests, for our particular project it takes
between 1 and 2 seconds to start the initial root watchman query, and thus
will always time out when started with Django.
My suggestion is that we increase the pywatchman client timeout higher
than 1 second
--
Ticket URL: <https://code.djangoproject.com/ticket/30361>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Jacobkg | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Jacobkg):
This is where I would consider setting a higher client timeout:
https://github.com/django/django/blob/99cfb28e995388db109e2f2ed15307f09945dda8/django/utils/autoreload.py#L390
We set a lower client timeout here as an example:
https://github.com/django/django/blob/99cfb28e995388db109e2f2ed15307f09945dda8/django/utils/autoreload.py#L390
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:1>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Daniel Hepper):
Jacob, just wanted to let you know that the two URLs you posted are
identical. I assume that was not intentional.
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:2>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tobias Kunze):
I had this issue occur on my machine, but I can't reproduce it. Is it
reproducible for you, Jacob? If so, could you check if
[https://github.com/rixx/django/tree/ticket_30361 this] solves the issue?
If it does, I'll submit it as PR.
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:3>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tobias Kunze):
* cc: Tobias Kunze (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:4>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Peter J. Farrell):
We are seeing the same issue especially the container starts (less often
if it's just a server reload). It appears the timeout on pywatchman needs
to be configurable from Django in this case as 1.0 seconds is too low in
certain circumstances.
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:5>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tobias Kunze):
A more generous default timeout might be a better choice than a
configuration option, in my opinion. In any case it would be great if
somebody who runs into this consistently could test
[https://github.com/rixx/django/commit/ec873d1448ebe922502c517c8ef90f441f1813ab
my patch] or provide one of their own.
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:6>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tobias Kunze):
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:7>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Jacob Green):
I run into the issue consistently on our codebase so I can repro it.
Tobias, how would you recommend that I test against your patch? I know how
to install Django from pypi but I don't know how to install it from a
fork.
Also I may be misreading but in the commit that you linked it seems that
you lowered the timeout to 0.2s instead of raising it above 1.0s. Perhaps
that was unintentional?
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:8>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by James Addison):
I've just found that adding a `.watchmanconfig` file to add the
`ignore_dirs` directive seemed to help (I assume this would allow watchman
to start more quickly). See
https://facebook.github.io/watchman/docs/config.html#ignore_dirs
{{{
{
"ignore_dirs": ["node_modules", ".envs", "bin", "lib", "include",
".git", ".idea", "templates", "static_assets"]
}
}}}
It certainly isn't a fix - but as mentioned in the Django docs, it's worth
investigating as a bandaid.
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:9>
Django
Reporter: Jacob Green | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 2.2
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Jacob Green):
I tried out this change (increasing timeout to 5 seconds) and that fixed
my issue:
https://github.com/django/django/compare/stable/2.2.x...Jacobkg:Jacobkg-
patch-1
I'm happy to submit this as a patch if there is interest
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:10>
Django
Reporter: Jacob Green | Owner: Jacob Green
Type: Bug | Status: assigned
Component: Utilities | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Jacob Green):
* owner: nobody => Jacob Green
* status: new => assigned
* has_patch: 0 => 1
* component: Uncategorized => Utilities
Comment:
I submitted a Pull Request, with the proposal that this be backported onto
2.2 since it improves the functionality of a feature released as part of
2.2
[https://github.com/django/django/pull/11263 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:11>
Django
Reporter: Jacob Green | Owner: Jacob Green
Type: Bug | Status: assigned
Component: Utilities | Version: 2.2
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:12>
Django
Reporter: Jacob Green | Owner: Jacob Green
Component: Utilities | Version: 2.2
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"ed3c59097a01ed3f32f8a8bed95307fb5c181251" ed3c590]:
{{{
#!CommitTicketReference repository=""
revision="ed3c59097a01ed3f32f8a8bed95307fb5c181251"
Fixed #30361 -- Increased the default timeout of watchman client to 5
seconds and made it customizable.
Made the default timeout of watchman client customizable via
DJANGO_WATCHMAN_TIMEOUT environment variable.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:13>
Django
Reporter: Jacob Green | Owner: Jacob Green
Type: Bug | Status: closed
Component: Utilities | Version: 2.2
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"e45763193fe496cefc6749c45e6a31e62f987d63" e4576319]:
{{{
#!CommitTicketReference repository=""
revision="e45763193fe496cefc6749c45e6a31e62f987d63"
[2.2.x] Fixed #30361 -- Increased the default timeout of watchman client
to 5 seconds and made it customizable.
Made the default timeout of watchman client customizable via
DJANGO_WATCHMAN_TIMEOUT environment variable.
Backport of ed3c59097a01ed3f32f8a8bed95307fb5c181251 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30361#comment:14>