[Django] #33380: Change runserver's auto-reloader to prefer (likely) user files when scanning for changes.
Django
scanning for changes.
-------------------------------------+-------------------------------------
Reporter: Keryn | Owner: nobody
Knight |
Type: New | Status: new
feature |
Component: Core | Version: dev
(Management commands) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Currently, because of the way `iter_modules_and_files` returns an LRU
cached frozenset, it executes (as far as I can tell) a maximum of twice
where everything downstream of that essentially executes on every `tick`,
so sorting further down is potentially a net detriment.
I'd like to propose changing that method to return a **tuple** (i.e.
similarly immutable) of ordered paths, by their likelihood of being
edited. This would I think reduce the number of stat calls in the common
scenario of a user's code changing, because those would be stat'd soonest
and `notify_file_changed` would fire to trigger a reload.
What I'm suggesting is something like (off the top of my head) the
following, give or take implementation details:
{{{
stdlib_location1 = dirname(os.__file__)
stdlib_location2 = dirname(dirname(logging.__file__)) # Based on my own
comments about having done this before, this isn't necessarily the same
path (e.g. pyenv shims)
django_location = os.path.dirname(os.path.dirname(django.__file__)) # and
again this may be a site-packages dir which is different
if is_django_path(resolved_path):
results.add((resolved_path, 10))
elif resolved_path.startswith(stdlib_location1, stdlib_location2,
django_location): # third party packages are potentially stored in these
locations
results.add((resolved_path, 5))
elif module in sys.builtin_module_names:
results.add((resolved_path, 30))
elif module in sys.stdlib_module_names: # eventually, it's new in py3.10
results.add((resolved_path, 20))
elif not isinstance(module.__spec__.loader, SourceFileLoaderOrWhatever):
# covers .so files, zip imports etc.
results.add((resolved_path, 40))
else:
results.add((resolved_path, 0))
}}}
and then ordering the `results` by the number rank, and returning a tuple
of those paths, in that order (again off the top of my head):
{{{
results = tuple(result for result, rank in sorted(results,
key=itemgetter(1)))
}}}
which would hopefully yield something like:
{{{
/path/to/myapp/views.py # (rank 0, local file, likely to change)
/another/path/django-livereloadish/livereloadish/middleware.py # (rank 0,
this is a symlink to my consolidated git repos, maybe changable)
/user/path/.direnv/python-3.9.5/src/enviable/enviable.py # (rank 0,
this'd be a pip editable install)
/user/path/.direnv/python-3.9.5/lib/python3.9/site-
packages/weasyprint/svg/path.py # (rank 5, within site packages,
potential for edits)
/user/path/.direnv/python-3.9.5/lib/python3.9/site-
packages/mistune/markdown.py # (rank 5, within site packages, potential
for edits)
/user/path/.direnv/python-3.9.5/lib/python3.9/site-
packages/django/db/models/utils.py # (rank 10, django path, may be edited
for debugging or development, but less likely in the scheme of things)
# ... ad infinitum for modules/files which rank as less likely to be
edited (dylibs, builtins, etc)
}}}
compared to what I currently might get, which looks something like this
(printing `filepath` within `tick()`):
{{{
/path/.asdf/installs/python/3.9.5/lib/python3.9/ctypes/macholib/framework.py
/path/.direnv/python-3.9.5/lib/python3.9/site-
packages/django/core/management/commands/runserver.py
/path/.asdf/installs/python/3.9.5/lib/python3.9/email/mime/base.py
/path/.asdf/installs/python/3.9.5/lib/python3.9/encodings/utf_16_le.py
# ... hundreds more stdlib/unlikely things
/another/path/django-livereloadish/livereloadish/patches.py
# ... a bunch more
/user/path/.direnv/python-3.9.5/src/enviable/enviable.py
# ... ad infinitum.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33380>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
scanning for changes.
-------------------------------------+-------------------------------------
Type: New feature | Status: new
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Tom Forbes (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/33380#comment:1>
Django
scanning for changes.
-------------------------------------+-------------------------------------
Type: New feature | Status: new
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
Hey Keryn — this sounds worth having a look at to me. 👍 Thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/33380#comment:2>
Django
scanning for changes.
-------------------------------------+-------------------------------------
| Hrushikesh Vaidya
Type: New feature | Status: assigned
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Hrushikesh Vaidya
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/33380#comment:3>
Django
scanning for changes.
-------------------------------------+-------------------------------------
| Hrushikesh Vaidya
Type: New feature | Status: assigned
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
* has_patch: 0 => 1
Comment:
An initial [https://github.com/django/django/pull/15418 PR]. It's not 100%
correct (some other non-project files manage to filter through), but all
of the user's project files are put at the start.
--
Ticket URL: <https://code.djangoproject.com/ticket/33380#comment:4>