[Django] #34787: The 'runserver' command doesn't work when run from an installed script on Windows
Django
Windows
-------------------------------------+-------------------------------------
Reporter: jp-larose | Owner: nobody
Type: Bug | Status: new
Component: Core | Version: 4.2
(Management commands) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-------------------------------------+-------------------------------------
My `manage.py` is as follows:
{{{
#!/usr/bin/env python
import os
import sys
def django_manage():
"""Function implementation of python manage.py"""
os.environ.setdefault("DJANGO_SETTINGS_MODULE",
"<project_package>.settings.dev")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
if __name__ == "__main__":
django_manage()
}}}
In my `pyproject.toml` I have:
{{{
[project.scripts]
"djm" = "<project_package>.manage:django_manage"
}}}
In Windows this generates a `djm.exe` file.
This allows me to save a few keystrokes when issuing commands from CLI.
And it works for most of the django commands. The only exception I've
encountered so far is with the `runserver` command.
It gives:
{{{
<project_path>\venv\Scripts\python.exe: Error while finding module
specification for '__main__' (ValueError: __main__.__spec__ is None)
}}}
After much debugging and tracing, I found where the issue lies. The
problem is in the `get_child_arguments` function in
`django/utils/autoreload.py`. When you flip the first two `if`-`elif`
blocks, everything works. That is, the check for `not py_script.exists()`
needs to come before the check for `getattr(__main__, "__spec__", None) is
not None`.
I'm not sure if this creates different problems, but it certainly fixes
the one I was having.
--
Ticket URL: <https://code.djangoproject.com/ticket/34787>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Windows
-------------------------------------+-------------------------------------
Reporter: jp-larose | Owner: nobody
Type: Bug | Status: new
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* Attachment
"Fix_for__runserver__command_when_run_from_an_exe_script_.patch" added.
Django
Windows
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Core (Management | Version: 4.2
commands) |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: David Smith (added)
* status: new => closed
* has_patch: 1 => 0
* resolution: => needsinfo
Comment:
Replying to [ticket:34787 Joël Larose]:
> It gives:
> {{{
> <project_path>\venv\Scripts\python.exe: Error while finding module
specification for '__main__' (ValueError: __main__.__spec__ is None)
> }}}
>
> After much debugging and tracing, I found where the issue lies. The
problem is in the `get_child_arguments` function in
`django/utils/autoreload.py`. When you flip the first two `if`-`elif`
blocks, everything works. That is, the check for `not py_script.exists()`
needs to come before the check for `getattr(__main__, "__spec__", None) is
not None`.
Can you provide the full stacktrace? I'm not sure how swapping these
branches can make a difference as the first one is protected against
`None` `__spec__`.
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:1>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: nobody
Component: Core (Management | Version: 4.2
commands) |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: needsinfo =>
Comment:
There's no stack trace produced. I even tried wrapping my code in try-
catch block, and it doesn't trap anything. I had to import pdb and trace
it to discover where the problem lies.
== Execution with the original code
Using the `exe`:
{{{
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django> djm
runserver
C:\Users\jplarose\Projects\green-rosewood\art-
django\venv\Scripts\python.exe: Error while finding module specification
for '__main__' (ValueError: __main__.__spec__ is None)
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django>
}}}
Using `python ...py`:
{{{
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django> python
.\greenrosewood_art\manage.py runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
August 21, 2023 - 00:11:37
Django version 4.2.4, using settings 'greenrosewood_art.site.settings.dev'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django>
}}}
Using `python -m`:
{{{
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django> python -m
greenrosewood_art.manage runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
August 21, 2023 - 00:14:36
Django version 4.2.4, using settings 'greenrosewood_art.site.settings.dev'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django>
}}}
== Execution with fixed code (i.e. `if` blocks flipped):
Using the `exe`:
{{{
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django> djm
runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
August 21, 2023 - 00:17:46
Django version 4.2.4, using settings 'greenrosewood_art.site.settings.dev'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django>
}}}
Using `python ...py`:
{{{
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django> python
.\greenrosewood_art\manage.py runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
August 21, 2023 - 00:19:58
Django version 4.2.4, using settings 'greenrosewood_art.site.settings.dev'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django>
}}}
Using `python -m`:
{{{
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django> python -m
greenrosewood_art.manage runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
August 21, 2023 - 00:19:09
Django version 4.2.4, using settings 'greenrosewood_art.site.settings.dev'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
(venv) PS C:\Users\jplarose\Projects\green-rosewood\art-django>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:2>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: nobody
Type: Bug | Status: new
Component: Core (Management | Version: 4.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce):
Hi Joël, thank you for this report!
Just looking at what changes might be related and found these: #31716
#32314. As not many of us have a Windows machine, can I be cheeky and ask
for you to check if this was working on a previous version of Django (to
rule out if this is a regression)? I would check 3.0 and 3.1. If it's
never been working then maybe a small sample project would also help to
make sure I get the file structure correct 💜
I'm still trying to wrap my head around it but it confuses me that we're
inside the if of `getattr(__main__, "__spec__", None) is not None` and yet
the error you seem to be seeing is `ValueError: __main__.__spec__ is None`
🤔
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:3>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: nobody
Type: Bug | Status: new
Component: Core (Management | Version: 4.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Joël Larose):
I created a minimal Django project with a `pyproject.toml` file to create
the custom script `exe`. I set the various versions of Django as optional
dependencies to make it easy to switch between the versions without having
to edit the file each time.
Here's a summary of the results:
- Django 3.0:
- {{{C:\Python\v311\python.exe: can't open file
'C:\\Users\\jplarose\\Projects\\django-windows-mvp\\venv\\Scripts\\djm':
[Errno 2] No such file or directory}}}
- Django 3.1 and 3.2:
- Works properly
- Django 4.0, 4.1, and 4.2:
- {{{C:\Users\jplarose\Projects\django-windows-
mvp\venv\Scripts\python.exe: Error while finding module specification for
'__main__' (ValueError: __main__.__spec__ is None)}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:4>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: nobody
Type: Bug | Status: new
Component: Core (Management | Version: 4.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* Attachment "pyproject.toml" added.
--
Ticket URL: <https://code.djangoproject.com/ticket/34787>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: nobody
Type: Bug | Status: new
Component: Core (Management | Version: 4.2
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Joël Larose):
One observation is that the system exits here:
{{{
def run_with_reloader(main_func, *args, **kwargs):
signal.signal(signal.SIGTERM, lambda *args: sys.exit(0))
try:
if os.environ.get(DJANGO_AUTORELOAD_ENV) == "true":
reloader = get_reloader()
logger.info(
"Watching for file changes with %s",
reloader.__class__.__name__
)
start_django(reloader, main_func, *args, **kwargs)
else:
exit_code = restart_with_reloader()
sys.exit(exit_code) # <--- Exits here
except KeyboardInterrupt:
pass
}}}
I added a pdb.set_trace() call in main(), here's a trace that might help:
{{{
(venv) PS C:\Users\jplarose\Projects\django-windows-mvp> djm runserver
> c:\users\jplarose\projects\django-windows-
mvp\django_windows_mvp\manage.py(10)main()
-> os.environ.setdefault('DJANGO_SETTINGS_MODULE',
'django_windows_mvp.settings')
(Pdb) b django/utils/autoreload.py:674
Breakpoint 1 at c:\users\jplarose\projects\django-windows-mvp\venv\lib
\site-packages\django\utils\autoreload.py:674
(Pdb) c
C:\Users\jplarose\Projects\django-windows-mvp\venv\Scripts\python.exe:
Error while finding module specification for '__main__' (ValueError:
__main__.__spec__ is None)
> c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\utils\autoreload.py(674)run_with_reloader()
-> sys.exit(exit_code)
(Pdb) where
<frozen runpy>(198)_run_module_as_main()
<frozen runpy>(88)_run_code()
c:\users\jplarose\projects\django-windows-
mvp\venv\scripts\djm.exe\__main__.py(7)<module>()
-> sys.exit(main())
c:\users\jplarose\projects\django-windows-
mvp\django_windows_mvp\manage.py(19)main()
-> execute_from_command_line(sys.argv)
c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\core\management\__init__.py(442)execute_from_command_line()
-> utility.execute()
c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\core\management\__init__.py(436)execute()
-> self.fetch_command(subcommand).run_from_argv(self.argv)
c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\core\management\base.py(412)run_from_argv()
-> self.execute(*args, **cmd_options)
c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\core\management\commands\runserver.py(74)execute()
-> super().execute(*args, **options)
c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\core\management\base.py(458)execute()
-> output = self.handle(*args, **options)
c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\core\management\commands\runserver.py(111)handle()
-> self.run(**options)
c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\core\management\commands\runserver.py(118)run()
-> autoreload.run_with_reloader(self.inner_run, **options)
> c:\users\jplarose\projects\django-windows-mvp\venv\lib\site-
packages\django\utils\autoreload.py(674)run_with_reloader()
-> sys.exit(exit_code)
(Pdb)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:5>
Django
Windows
-------------------------------------+-------------------------------------
| Boyce
Type: Bug | Status: assigned
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => assigned
* needs_better_patch: 0 => 1
* version: 4.2 => dev
* owner: nobody => Sarah Boyce
* has_patch: 0 => 1
* stage: Unreviewed => Accepted
Comment:
Thank you for the extra input Joël! I really appreciate it
I suspect this might be a regression introduced in #32669. Accepted the
ticket 👍
I have a draft PR: https://github.com/django/django/pull/17203
Hoping for some testing to confirm whether this works (will mark as "Patch
need improvement" until we're happy it's working as expected).
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:6>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: Sarah
| Boyce
Type: Bug | Status: assigned
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Joël Larose):
Thanks Sarah! Your code fixes the issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:7>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: Sarah
| Boyce
Type: Bug | Status: assigned
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
Comment:
Thank you for testing!
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:8>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: Sarah
| Boyce
Type: Bug | Status: assigned
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:9>
Django
Windows
-------------------------------------+-------------------------------------
Reporter: Joël Larose | Owner: Sarah
| Boyce
Component: Core (Management | Version: dev
commands) |
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"f6ed2c36ddf35bcc32c912123a4eb6fe0a6bfc6a" f6ed2c3]:
{{{
#!CommitTicketReference repository=""
revision="f6ed2c36ddf35bcc32c912123a4eb6fe0a6bfc6a"
Fixed #34787 -- Fixed autoreloader crash when run from installed script on
Windows.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34787#comment:10>