[Django] #34204: Django cannot load when Python is compiled with --without-doc-strings enabled
Django
enabled
-----------------------------------------+------------------------
Reporter: Jon Janzen | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 4.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
I'm not sure that this is even a supported configuration for Django, but
CPython's `./configure` script offers an option called
[https://docs.python.org/3/using/configure.html#cmdoption-without-doc-
strings --without-doc-strings] which improves memory utilization slightly.
Quoting the relevant portion from CPython docs:
> --without-doc-strings
> Disable static documentation strings to reduce the memory footprint
(enabled by default). Documentation strings defined in Python are not
affected.
I have an use-case where I need to deploy a Django service on a device
with low available memory. I'd like to disable built-in doc strings as
part of an overall strategy to reduce memory but compiling CPython with
that option and running the django service crashes:
{{{
File ".../asgi.py", line 3, in <module>
from django.core.asgi import get_asgi_application
File ".../lib/python3.11/site-packages/django/core/asgi.py", line 2, in
<module>
from django.core.handlers.asgi import ASGIHandler
File ".../lib/python3.11/site-packages/django/core/handlers/asgi.py",
line 11, in <module>
from django.core.handlers import base
File ".../lib/python3.11/site-packages/django/core/handlers/base.py",
line 11, in <module>
from django.urls import get_resolver, set_urlconf
File ".../lib/python3.11/site-packages/django/urls/__init__.py", line 1,
in <module>
from .base import (
File ".../lib/python3.11/site-packages/django/urls/base.py", line 8, in
<module>
from .exceptions import NoReverseMatch, Resolver404
File ".../lib/python3.11/site-packages/django/urls/exceptions.py", line
1, in <module>
from django.http import Http404
File ".../lib/python3.11/site-packages/django/http/__init__.py", line 2,
in <module>
from django.http.request import (
File ".../lib/python3.11/site-packages/django/http/request.py", line 8,
in <module>
from django.core import signing
File ".../lib/python3.11/site-packages/django/core/signing.py", line 43,
in <module>
from django.utils.crypto import constant_time_compare, salted_hmac
File ".../lib/python3.11/site-packages/django/utils/crypto.py", line 85,
in <module>
if func_supports_parameter(hashlib.md5, 'usedforsecurity'):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/django/utils/inspect.py", line
74, in func_supports_parameter
return any(param.name == name for param in
_get_callable_parameters(func))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/django/utils/inspect.py", line
16, in _get_callable_parameters
return _get_func_parameters(func, remove_first=is_method)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/django/utils/inspect.py", line 7,
in _get_func_parameters
parameters = tuple(inspect.signature(func).parameters.values())
^^^^^^^^^^^^^^^^^^^^^^^
File ".../cpython/Lib/inspect.py", line 3270, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../cpython/Lib/inspect.py", line 3018, in from_callable
return _signature_from_callable(obj, sigcls=cls,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../cpython/Lib/inspect.py", line 2510, in
_signature_from_callable
return _signature_from_builtin(sigcls, obj,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../cpython/Lib/inspect.py", line 2317, in _signature_from_builtin
raise ValueError("no signature found for builtin {!r}".format(func))
ValueError: no signature found for builtin <built-in function openssl_md5>
}}}
(irrelevant information lightly censored)
Looking through this stack trace the problem looks to be
[https://github.com/django/django/blob/d10c7bfe56f025ccc690721c9f13e7029b777b9c/django/utils/crypto.py#L80-L92
some code] that executes at module import to determine whether or not md5
supports the `usedforsecurity` parameter.
If this ticket is accepted I'm happy to put up a PR to fix this, I've
included my proposed solution on this ticket to save a roundtrip in
discussion:
{{{
diff --git a/django/utils/inspect.py b/django/utils/inspect.py
index 28418f7312..d8622a22df 100644
--- a/django/utils/inspect.py
+++ b/django/utils/inspect.py
@@ -70,4 +70,12 @@ def method_has_no_args(meth):
def func_supports_parameter(func, name):
- return any(param.name == name for param in
_get_callable_parameters(func))
+ try:
+ callable_parameters = _get_callable_parameters(func)
+ except ValueError:
+ # When Python is compiled with the --without-doc-strings
+ # argument to ./configure the signatures for built-in
+ # functions are not available. In such a case, be
+ # conservative and assume no:
+ return False
+ return any(param.name == name for param in callable_parameters)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34204>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
enabled
Reporter: Jon Janzen | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 4.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by Jon Janzen:
Old description:
New description:
(irrelevant information lightly censored)
An easy way to repro is to follow this script:
{{{
git clone https://github.com/python/cpython.git
cd cpython
git checkout 3.11 # or whatever version you have that already has django
installed
./configure --without-doc-strings
make -j
./python
>>> import django.utils.crypto
*crash*
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34204#comment:1>
Django
Old description:
New description:
(irrelevant information lightly censored)
./python -c "import django.utils.crypto"
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34204#comment:2>
Django
enabled
-------------------------------+--------------------------------------
Reporter: Jon Janzen | Owner: nobody
Component: Core (Other) | Version: 4.1
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
* status: new => closed
* resolution: => invalid
* component: Uncategorized => Core (Other)
Comment:
Thanks for this report, however it seems to be an issue in Python not in
Django itself. According to Python's docs, the
[https://docs.python.org/3/using/configure.html?highlight=without%20doc%20strings
#cmdoption-without-doc-strings --without-doc-strings] option should not
affect builtins. Moreover, we check signatures in other places, I don't
think it's worth the extra complexity.
--
Ticket URL: <https://code.djangoproject.com/ticket/34204#comment:3>
Django
enabled
-------------------------------+--------------------------------------
Reporter: Jon Janzen | Owner: nobody
Component: Core (Other) | Version: 4.1
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
* status: closed => new
* resolution: invalid =>
Comment:
Hey Mariusz,
> According to Python's docs, the --without-doc-strings option should not
affect builtins.
I must be missing something because as I read the docs you linked it fails
to mention this limitation, but that is not the same thing as saying that
it should not affect builtins.
In fact, when you read the CPython source code there are a few places that
make it more explicit. For example, the tests for parsing the signatures
of builtins are ''disabled'' when docstrings are disabled:
Additionally, there is some commentary in-lined into the `inspect` module
about this:
I'll open a ticket with CPython when I have a chance to ask them to
document the limitation, but this is an intentional part of compiling with
`--without-doc-strings`.
> Moreover, we check signatures in other places, I don't think it's worth
the extra complexity.
Is this referring to the specific solution I offered or the concept of
fixing this issue overall?
Sorry if reopening isn't the proper procedure, I'm still trying to learn
my way around Django's process
--
Ticket URL: <https://code.djangoproject.com/ticket/34204#comment:4>
Django
enabled
-------------------------------+--------------------------------------
Reporter: Jon Janzen | Owner: nobody
Type: Uncategorized | Status: new
Component: Core (Other) | Version: 4.1
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Comment (by Jon Janzen):
CPython issue xref: https://github.com/python/cpython/issues/100150
--
Ticket URL: <https://code.djangoproject.com/ticket/34204#comment:5>
Django
enabled
------------------------------+--------------------------------------
Reporter: Jon Janzen | Owner: nobody
Component: Core (Other) | Version: 4.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* status: new => closed
* type: Uncategorized => New feature
* resolution: => wontfix
Comment:
> Is this referring to the specific solution I offered or the concept of
fixing this issue overall?
Almost all hooks in the `django.utils.inspect` module are affected, not
just `func_supports_parameter()`. Adding official support for `CPython`
compiled with `--without-doc-strings` would required catching `ValueError`
in all of them. I still don't think it's worth the extra complexity and it
would be clunky and confusing for users as we cannot return false
positives/negatives in all these places.
I appreciate you'd like to reopen the ticket, but please
[https://docs.djangoproject.com/en/stable/internals/contributing/triaging-
tickets/#closing-tickets follow the triaging guidelines with regards to
wontfix tickets] and take this to DevelopersMailingList, where you'll
reach a wider audience and see what other think,
--
Ticket URL: <https://code.djangoproject.com/ticket/34204#comment:6>