[Django] #30364: python3 compileall not working anymore with Django 2.2
Django
-------------------------------------+-------------------------------------
Reporter: Sascha P. | Owner: nobody
Type: Bug | Status: new
Component: | Version: 2.2
Uncategorized | Keywords: compileall, Django
Severity: Normal | 2.2, encoding
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Hi,
We are shipping python in docker containers in bytecode for legal purposes
and obfuscation. With the new Django 2.2 this stopped working, due to an
encoding issue. To reproduce this error we receive, we created a minimal
dockerfile:
{{{
FROM python:alpine3.6
RUN apk upgrade --no-cache && \
pip install Django==2.2.0 && \
django-admin startproject mysite && \
python3 mysite/manage.py makemigrations && \
python3 -m compileall -b -f mysite && \
find mysite -type f -name '*.py' | xargs rm && \
mv mysite/manage.pyc mysite/manage.py && \
python3 mysite/manage.py makemigrations
CMD /bin/sh
}}}
You will get this error:
{{{
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/django/template/utils.py",
line 66, in __getitem__
return self._engines[alias]
KeyError: 'django'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "mysite/manage.py", line 21, in <module>
File "mysite/manage.py", line 17, in main
File "/usr/local/lib/python3.6/site-
packages/django/core/management/__init__.py", line 381, in
execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-
packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.6/site-
packages/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.6/site-
packages/django/core/management/base.py", line 361, in execute
self.check()
File "/usr/local/lib/python3.6/site-
packages/django/core/management/base.py", line 390, in check
include_deployment_checks=include_deployment_checks,
File "/usr/local/lib/python3.6/site-
packages/django/core/management/base.py", line 377, in _run_checks
return checks.run_checks(**kwargs)
File "/usr/local/lib/python3.6/site-
packages/django/core/checks/registry.py", line 72, in run_checks
new_errors = check(app_configs=app_configs)
File "/usr/local/lib/python3.6/site-
packages/django/contrib/admin/checks.py", line 80, in check_dependencies
for engine in engines.all():
File "/usr/local/lib/python3.6/site-packages/django/template/utils.py",
line 90, in all
return [self[alias] for alias in self]
File "/usr/local/lib/python3.6/site-packages/django/template/utils.py",
line 90, in <listcomp>
return [self[alias] for alias in self]
File "/usr/local/lib/python3.6/site-packages/django/template/utils.py",
line 81, in __getitem__
engine = engine_cls(params)
File "/usr/local/lib/python3.6/site-
packages/django/template/backends/django.py", line 23, in __init__
options.setdefault('file_charset', settings.FILE_CHARSET)
File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py",
line 134, in FILE_CHARSET
stack = traceback.extract_stack()
File "/usr/local/lib/python3.6/traceback.py", line 207, in extract_stack
stack = StackSummary.extract(walk_stack(f), limit=limit)
File "/usr/local/lib/python3.6/traceback.py", line 358, in extract
f.line
File "/usr/local/lib/python3.6/traceback.py", line 282, in line
self._line = linecache.getline(self.filename, self.lineno).strip()
File "/usr/local/lib/python3.6/linecache.py", line 16, in getline
lines = getlines(filename, module_globals)
File "/usr/local/lib/python3.6/linecache.py", line 47, in getlines
return updatecache(filename, module_globals)
File "/usr/local/lib/python3.6/linecache.py", line 137, in updatecache
lines = fp.readlines()
File "/usr/local/lib/python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8d in position 4:
invalid start byte
}}}
If you use Django==2.1.8 instead you will get the expected "No changes
detected" and everything works fine. There has been some work around
"FILE_CHARSET" with Django 2.2.0, so maybe thats where this comes from.
We normally always use the newest Django version, yet with this one we had
to pin the version to 2.1.8.
If this is not a bug caused by Django, then Im sorry for reporting this,
yet I hope that someone could point me to the right direction.
Best regards
Sascha
--
Ticket URL: <https://code.djangoproject.com/ticket/30364>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: Sascha P. | Owner: nobody
Component: Uncategorized | Version: 2.2
Severity: Normal | Resolution: invalid
Keywords: compileall, Django | Triage Stage:
2.2, encoding | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
Renaming a .pyc file to .py is not anything that is supposed to work.
--
Ticket URL: <https://code.djangoproject.com/ticket/30364#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Sascha P. | Owner: nobody
Component: Uncategorized | Version: 2.2
Severity: Normal | Resolution: invalid
Keywords: compileall, Django | Triage Stage:
2.2, encoding | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Bell):
The deprecation check for settings.FILE_CHARSET added in #29817 uses
`traceback.extract_stack()` to find out where the setting is being
accessed from. When a project's .py files have been deleted, leaving only
the .pyc files, `extract_stack()` throws an exception as it tries to read
the .pyc file.
Running `manage.py check` will tickle this "bug" when it checks the
template engines.
--
Ticket URL: <https://code.djangoproject.com/ticket/30364#comment:2>
Django
Reporter: Sascha P. | Owner: nobody
Component: Uncategorized | Version: 2.2
Severity: Normal | Resolution: invalid
Keywords: compileall, Django | Triage Stage:
2.2, encoding | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Florian Apolloner):
That is not exactly true, at least not in this case. In this case the
issue is caused by the fact that they rename the pyc file to py which
extract_stack then fails on because a py file is supposed to be readable…
--
Ticket URL: <https://code.djangoproject.com/ticket/30364#comment:3>
Django
Reporter: Sascha P. | Owner: nobody
Component: Uncategorized | Version: 2.2
Severity: Normal | Resolution: invalid
Keywords: compileall, Django | Triage Stage:
2.2, encoding | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Florian Apolloner (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/30364#comment:4>
Django
Reporter: Sascha P. | Owner: nobody
Component: Uncategorized | Version: 2.2
Severity: Normal | Resolution: invalid
Keywords: compileall, Django | Triage Stage:
2.2, encoding | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sascha P.):
Thanks at everyone. At least I know now into which direction to
investigate.
--
Ticket URL: <https://code.djangoproject.com/ticket/30364#comment:5>