[Django] #33139: Using IPython for the manage.py REPL interface clobbers __main__, where the normal python REPL doesn't.
Django
normal python REPL doesn't.
-------------------------------------+-------------------------------------
Reporter: Keryn | Owner: Keryn Knight
Knight |
Type: Bug | Status: assigned
Component: Core | Version: dev
(Management commands) |
Severity: Normal | Keywords: ipython shell
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
The only thing I can find of related note is #30588 which observes a
similar thing but around auto-reloading*
Given the following file, which is ... I believe it has been called
''esoteric'' but works:
{{{
import django
from django.conf import settings
from django.http import HttpResponse
from django.urls import path
if not settings.configured:
settings.configure(
SECRET_KEY="?",
DEBUG=True,
INSTALLED_APPS=(),
ROOT_URLCONF=__name__,
)
django.setup()
def test(request):
return HttpResponse('test')
urlpatterns = [
path("test/", test, name="test"),
path("best/", test, name="test"),
]
if __name__ == "__main__":
from django.core import management
management.execute_from_command_line()
else:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
}}}
it is possible to reverse those url patterns, like so:
{{{
$ python whypython.py shell -ipython
Python 3.9.5 ...
>>> __name__
'builtins'
>>> import sys
>>> sys.modules['__main__']
<module '__main__' from '/path/to/whypython.py'>
>>> dir(sys.modules['__main__'])
['HttpResponse', ..., 'urlpatterns']
>>> sys.modules['builtins']
<module 'builtins' (built-in)>
>>> from django.urls import reverse
>>> reverse('test')
'/best/'
>>>
}}}
But doing the same via the IPython support:
{{{
$ python whypython.py shell -iipython
Python 3.9.5 ...
IPython 7.24.1 ...
In [1]: __name__
Out[1]: '__main__'
In [2]: import sys
In [3]: sys.modules['__main__']
Out[3]: <module '__main__'>
In [4]: dir(sys.modules['__main__'])
Out[4]:
['In',
'Out',
...
'quit',
'sys']
In [5]: sys.modules['builtins']
Out[5]: <module 'builtins' (built-in)>
In [6]: from django.urls import reverse
In [7]: reverse('test')
...
ImproperlyConfigured: The included URLconf '__main__' does not appear to
have any patterns in it [...]
}}}
This is because `django.core.management.commands.shell.Command.ipython`
invokes `start_python` without supplying a `user_ns`, which means that
down in
`IPython.core.interactiveshell.InteractiveShell.prepare_user_module` this
runs:
{{{
if user_module is None:
user_module = types.ModuleType("__main__",
doc="Automatically created module for IPython interactive
environment")
}}}
and then `IPython.core.interactiveshell.InteractiveShell.init_sys_modules`
patches that oddness in. I don't know ''why'' it does so, particularly.
The "''fix''" is to change the Django invocation to:
{{{
start_ipython(argv=[], user_ns={'__name__': '__ipython_main__'})
}}}
which leaves `__main__` in peace, and inserts a `DummyMod` with that name
into the sys.modules table. As far as I can tell, both are treated by
IPython the same ... just a big `__dict__` to patch things into.
I suppose an argument could be made for checking if `__main__` is in
`sys.modules` and only set the `user_ns` if it is.
You'd think that supplying `user_module=...` would also be an effective
fix, but alas, `IPython.terminal.ipapp.TerminalIPythonApp.init_shell`
doesn't pass that along, only `user_ns`. I don't know if it's possible to
pass it along ''somehow'' because of the amount of magic in IPython
(traits, observes, etc)
Thus, using `user_ns`, it all works:
{{{
In [1]: __name__
Out[1]: '__ipython_main__'
In [2]: import sys
In [3]: sys.modules['__main__']
Out[3]: <module '__main__' from '/path/to/whypython.py'>
In [4]: sys.modules[__name__]
Out[4]: <IPython.core.interactiveshell.DummyMod at 0x10659da90>
In [5]: dir(sys.modules['__main__'])
Out[5]:
['HttpResponse',
...
'urlpatterns']
In [6]: dir(sys.modules[__name__])
['In',
'Out',
...
'quit',
'sys']
In [7]: from django.urls import reverse
In [8]: reverse('test')
Out[8]: '/best/'
}}}
I've got little concrete idea about the knock-on effects on changing it --
I don't ''feel'' like there should be any, because as I mentioned, I think
they're both just used as glorified dicts. But it does change the global
`__name__` and I don't know how to avoid that off the top of my head, I
only know that I can't think of many reasons to be checking `__name__` in
the REPL, beyond pulling it from `sys.modules` for reasons (and there I
think replacing/shadowing the real one is a very weird move for IPython to
have made).
For historical reference, here's the pdb `where` output to get to the bit
which triggers a change in behaviour:
{{{
/path/to/whypython.py(29)<module>()
-> management.execute_from_command_line()
/path/to/django/core/management/__init__.py(419)execute_from_command_line()
-> utility.execute()
/path/to/django/core/management/__init__.py(413)execute()
-> self.fetch_command(subcommand).run_from_argv(self.argv)
/path/to/django/core/management/base.py(363)run_from_argv()
-> self.execute(*args, **cmd_options)
/path/to/django/core/management/base.py(407)execute()
-> output = self.handle(*args, **options)
/path/to/django/core/management/commands/shell.py(112)handle()
-> return getattr(self, shell)(options)
/path/to/django/core/management/commands/shell.py(36)ipython()
-> start_ipython(argv=[], user_ns={'__name__': '__ipython_main__'})
/path/to/site-packages/IPython/__init__.py(126)start_ipython()
-> return launch_new_instance(argv=argv, **kwargs)
/path/to/site-
packages/traitlets/config/application.py(844)launch_instance()
-> app.initialize(argv)
/path/to/site-packages/traitlets/config/application.py(87)inner()
-> return method(app, *args, **kwargs)
/path/to/site-packages/IPython/terminal/ipapp.py(317)initialize()
-> self.init_shell()
/path/to/site-packages/IPython/terminal/ipapp.py(331)init_shell()
-> self.shell = self.interactive_shell_class.instance(parent=self,
/path/to/site-packages/traitlets/config/configurable.py(537)instance()
-> inst = cls(*args, **kwargs)
/path/to/site-
packages/IPython/terminal/interactiveshell.py(525)__init__()
-> super(TerminalInteractiveShell, self).__init__(*args, **kwargs)
/path/to/site-packages/IPython/core/interactiveshell.py(647)__init__()
-> self.init_create_namespaces(user_module, user_ns)
/path/to/site-
packages/IPython/core/interactiveshell.py(1239)init_create_namespaces()
-> self.user_module, self.user_ns = self.prepare_user_module(user_module,
user_ns)
> /path/to/site-
packages/IPython/core/interactiveshell.py(1307)prepare_user_module()
}}}
Assigning it to myself on the off-chance it is accepted.
----
(* and I ''think'' in the process links to the wrong `start_ipython`, to
no detriment. The actual one used is `IPython.start_ipython` rather than
`IPython.testing.globalipapp.start_ipython`, whose implementation is
different)
--
Ticket URL: <https://code.djangoproject.com/ticket/33139>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
normal python REPL doesn't.
-------------------------------------+-------------------------------------
| Knight
Type: Bug | Status: closed
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution: wontfix
Keywords: ipython shell | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => wontfix
Comment:
Thanks for the ticket. Closing as "wontfix" based on discussion in #19737.
Feel free to write to the DevelopersMailingList if you disagree with the
conclusions of that ticket.
--
Ticket URL: <https://code.djangoproject.com/ticket/33139#comment:1>