[Django] #36544: In some import sequence ./manage.py raises "populate() isn't reentrant" hiding the original error

[Django]

#36544: In some import sequence ./manage.py raises "populate() isn't reentrant"
hiding the original error
-------------------------------------+-------------------------------------
Reporter: living-dev | Type:
| Uncategorized
Status: new | Component: Core
| (Management commands)
Version: 5.2 | Severity: Normal
Keywords: populate reentrant | Triage Stage:
manage.py | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Hi,

I often add other command line entry points to my project, as a
consequence, I ensure that whenever some code use my project code, all the
django part is initialized if not already done. This can lead to a import
sequence by manage.py that hides the real error.

Here is a reproduction scenario:

{{{
$ python -m venv bug
$ cd bug/
$ . ./bin/activate
$ pip install django==5.2.5
$ django-admin startproject bug
$ cd bug
$ ./manage.py startapp an_app
$ ./manage.py startapp bad_import
}}}

Add "an_app" and "bad_import" to INSTALLED_APPS:

{{{
INSTALLED_APPS = [
...
'an_app',
'bad_import',
]
}}}

Here the automatic django initialization code:

{{{
$ cat > bug/__init__.py << EOF
import os

import django

from django.conf import settings
from django.apps import apps
if not settings.configured and not apps.loading:
os.environ['DJANGO_SETTINGS_MODULE'] = 'bug.settings'
django.setup()
EOF
}}}

And a custom entry point under bug project:

{{{
$ cat > bug/cli.py << EOF
from django.conf import settings
if __name__ == '__main__':
print('My CLI entry point:', settings.configured)
EOF
}}}

At this point both cli and `manage.py` are working:

{{{

$ python -m bug.cli
My CLI entry point: True
$ ./manage.py
Type 'manage.py help <subcommand>' for help on a specific subcommand.
...
}}}

Now we introduce a bad import line because of typo error in the models:

{{{
$ echo 'import an_app.model' >> bad_import/models.py
}}}

When using our cli entry point, the error is clear:

{{{
$ python -m bug.cli
Traceback (most recent call last):
...
File "/home/debian/Support/django/bug-20250807/bug/bug/bug/__init__.py",
line 9, in <module>
django.setup()
...
File
"/home/debian/Support/django/bug-20250807/bug/bug/bad_import/models.py",
line 3, in <module>
import an_app.model
ModuleNotFoundError: No module named 'an_app.model'
}}}

But when we use manage.py, the error is hidden:

{{{
$ ./manage.py
Traceback (most recent call last):
...
File "/home/debian/Support/django/bug-20250807/bug/lib/python3.11/site-
packages/django/core/management/__init__.py", line 416, in execute
django.setup()
File "/home/debian/Support/django/bug-20250807/bug/lib/python3.11/site-
packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/debian/Support/django/bug-20250807/bug/lib/python3.11/site-
packages/django/apps/registry.py", line 83, in populate
raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant
}}}

In fact the error is memorized in
`django/core/management/__init__.py:381`:

{{{
try:
settings.INSTALLED_APPS
except ImproperlyConfigured as exc:
self.settings_exception = exc
except ImportError as exc:
self.settings_exception = exc
}}}

But we finish on this part, that triggers the reentrant populate
`django/core/management/__init__.py:414`:

{{{
# In all other cases, django.setup() is required to succeed.
else:
django.setup()
}}}

I think it could be simply resolved by propagating the memorized exception
if `django.setup()` fails, so the trace is more explicit:

{{{
# In all other cases, django.setup() is required to succeed.
else:
try:
django.setup()
except Exception as exc:
raise exc from self.settings_exception
}}}

In this case the stack trace can help as it shows both the original error
and that we also call `django.setup()` twice:

{{{
$ ./manage.py
Traceback (most recent call last):
File "/home/debian/Support/django/bug-20250807/bug/lib/python3.11/site-
packages/django/core/management/__init__.py", line 382, in execute
settings.INSTALLED_APPS
...
File "/home/debian/Support/django/bug-20250807/bug/bug/bug/__init__.py",
line 9, in <module>
django.setup()
...
File
"/home/debian/Support/django/bug-20250807/bug/bug/bad_import/models.py",
line 3, in <module>
from an_app.model import Model
ModuleNotFoundError: No module named 'an_app.model'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/debian/Support/django/bug-20250807/bug/bug/./manage.py",
line 22, in <module>
main()
...
File "/home/debian/Support/django/bug-20250807/bug/lib/python3.11/site-
packages/django/core/management/__init__.py", line 417, in execute
django.setup()
...
File "/home/debian/Support/django/bug-20250807/bug/lib/python3.11/site-
packages/django/apps/registry.py", line 83, in populate
raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant
}}}

Best Regards,
--
Ticket URL: <https://code.djangoproject.com/ticket/36544>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36544: In some import sequence ./manage.py raises "populate() isn't reentrant"
hiding the original error
-------------------------------------+-------------------------------------

Reporter: living-dev | Owner: (none)
Type: Bug | Status: closed
Component: Core (Management | Version: 5.2
commands) |
Severity: Normal | Resolution: duplicate

Keywords: populate reentrant | Triage Stage:

manage.py ImportError | Unreviewed

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* keywords: populate reentrant manage.py => populate reentrant manage.py
ImportError
* resolution: => duplicate
* status: new => closed
* type: Uncategorized => Bug

Comment:

Thanks for the report. The swallowing of `ImportError` regardless of
whether it emanated from a settings module is tracked in #32915. A clear
statement of the problem can also be found in #36422.
--
Ticket URL: <https://code.djangoproject.com/ticket/36544#comment:1>