Django 2.2 and the watchman reloader

[Claude Paroz]

Hi,

The new watchman-based autoreloader in Django 2.2 is a nice improvement. However, the main issue in my opinion is that there are no Debian/Ubuntu official packaging of watchman to my knowledge.
I've been able to compile it without difficulty, but I'm especially concerned about newcomers and non-experienced programmers which will be faced to a startup warning message and a rather heavy system load when they start developing with Django. Telling them they have to compile from sources to get rid of that is not very friendly at least.

Tom (Forbes), as the author of this refactoring, could you tell us if it would be much work to create a pyinotify reloader to fallback when watchman is not installed?

Claude

[Dan Davis]

Claude,

I've tested a Django based application on 2.2b1 without watchman on Windows, it does tell you about watchman, but it doesn't fail to run.  Apparently, it falls back to the old way of reloading.   Is this not the behavior on Debian/Ubuntu? 

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/23ecaa54-a148-4439-a16b-16937da9e934%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Claude Paroz]

Le jeudi 21 février 2019 19:48:31 UTC+1, Dan Davis a écrit :

Claude,

I've tested a Django based application on 2.2b1 without watchman on Windows, it does tell you about watchman, but it doesn't fail to run.  Apparently, it falls back to the old way of reloading.   Is this not the behavior on Debian/Ubuntu?

Yes it is. I would say this is still a slight regression in two ways:

- no messages told you the reload method was not optimal before. So now people will try to "fix" their system, more than before.

- for Debian-based systems, you could improve the reloading performance by installing system or pip packages in 30 seconds. Now you have to spend 30 minutes to search how watchman can be installed and to compile the package (+ you have to care yourself for any security issue).

Claude

[Claude Paroz]

I have a POC patch here: https://github.com/django/django/pull/11014

Claude

[Tom Forbes]

Hey Claude,

Thanks for your feedback on the feature, I fully agree with you. I think we should remove that warning message about the missing package. I will make a PR to do that.

Regarding creating another reloader: it should not be that difficult to do at all since we have all the other pieces in place. In theory it's just implementing a class with single generator method.

If people agree I would like to use the 'watchdog' package for this rather than the pyinotify library as it would be quicker to implement, a lot nicer to work with and is easier to test.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b059553a-e25e-4d90-beed-bf7e0f797305%40googlegroups.com.

[Claude Paroz]

Le jeudi 21 février 2019 21:43:43 UTC+1, Tom Forbes a écrit :

Hey Claude,

Thanks for your feedback on the feature, I fully agree with you. I think we should remove that warning message about the missing package. I will make a PR to do that.

I'm not completely sure it's a good idea to entirely remove the message. Maybe just telling the used reloader would be fine.

 

Regarding creating another reloader: it should not be that difficult to do at all since we have all the other pieces in place. In theory it's just implementing a class with single generator method.

If people agree I would like to use the 'watchdog' package for this rather than the pyinotify library as it would be quicker to implement, a lot nicer to work with and is easier to test.

++1, watchdog is better maintained, looks like pyinotify is dead. Tell me if you need help, even if you seems a lot more knowledgeable than me on the subject.

 

Thanks.

Claude

[Tom Forbes]

I have a small PR here to remove the “watchman unavailable” message, whilst keeping the one that specifies which reloader we are using: https://github.com/django/django/pull/11025.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/6de70f64-9bbb-4017-bd10-c85e94ae08d0%40googlegroups.com.

[Tom Forbes]

Just an update to this: I've made a WIP PR to add watchdog support here: https://github.com/django/django/pull/11480

I'd like to disable it on Windows and MacOS, and there are some tricky issues to resolve around how events are triggered (watchdog uses a number of threads internally, and we need to ensure only the main thread triggers the file changed notification), but otherwise it seems to be working OK locally.