[Django] #29027: file_move_safe error with SELinux
Django
------------------------------------------------+-------------------------
Reporter: bhargu | Owner: nobody
Type: Bug | Status: new
Component: File uploads/storage | Version: 1.11
Severity: Normal | Keywords: selinux
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+-------------------------
PermissionError on upload of large files (large enough to not use the in-
memory file class) when SELinux is in enforce mode.
The error happens after the file is copied to the destination folder. As
seen in the trace below, when copystat is called from file_move_safe,
shutil tries to set attributes with setxattr. But this is failing when
SELinux is enabled. I noticed on temporarily disabling SELinux that the
files which are successfully uploaded now have a context of 'httpd_tmp_t'
as opposed to the context of 'httpd_sys_rw_content_t' which the directory
is configured with.
So, when SELinux is enabled, setxattr is failing when the context of the
file is 'httpd_sys_rw_content_t'.
{{{
Traceback (most recent call last):
File "/home/portal/venv/lib/python3.6/site-
packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/home/portal/venv/lib/python3.6/site-
packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/home/portal/venv/lib/python3.6/site-
packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/portal/venv/lib/python3.6/site-
packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
File "/home/portal/venv/lib/python3.6/site-
packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/home/portal/project/assignments/views/views.py", line 270, in
edit_answer
instance.answerfile_set.create(file=upload, file_type=file)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/fields/related_descriptors.py", line 653, in
create
return super(RelatedManager, self.db_manager(db)).create(**kwargs)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/query.py", line 394, in create
obj.save(force_insert=True, using=self.db)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/base.py", line 808, in save
force_update=force_update, update_fields=update_fields)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/base.py", line 838, in save_base
updated = self._save_table(raw, cls, force_insert, force_update,
using, update_fields)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/base.py", line 924, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk,
raw)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/base.py", line 963, in _do_insert
using=using, raw=raw)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/query.py", line 1076, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 1111, in execute_sql
for sql, params in self.as_sql():
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 1064, in as_sql
for obj in self.query.objs
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 1064, in <listcomp>
for obj in self.query.objs
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 1063, in <listcomp>
[self.prepare_value(field, self.pre_save_val(field, obj)) for field in
fields]
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 1013, in pre_save_val
return field.pre_save(obj, add=True)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/fields/files.py", line 296, in pre_save
file.save(file.name, file.file, save=False)
File "/home/portal/venv/lib/python3.6/site-
packages/django/db/models/fields/files.py", line 94, in save
self.name = self.storage.save(name, content,
max_length=self.field.max_length)
File "/home/portal/venv/lib/python3.6/site-
packages/django/core/files/storage.py", line 54, in save
return self._save(name, content)
File "/home/portal/venv/lib/python3.6/site-
packages/django/core/files/storage.py", line 338, in _save
file_move_safe(content.temporary_file_path(), full_path)
File "/home/portal/venv/lib/python3.6/site-
packages/django/core/files/move.py", line 76, in file_move_safe
copystat(old_file_name, new_file_name)
File "/usr/lib64/python3.6/shutil.py", line 225, in copystat
_copyxattr(src, dst, follow_symlinks=follow)
File "/usr/lib64/python3.6/shutil.py", line 165, in _copyxattr
os.setxattr(dst, name, value, follow_symlinks=follow_symlinks)
PermissionError: [Errno 13] Permission denied:
'/home/portal/project/media/submission/8/69_424/10465_fxjc_1_27691_uljy_1_46853.png'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29027>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: bhargu | Owner: nobody
Type: Bug | Status: new
uploads/storage |
Severity: Normal | Resolution:
Keywords: selinux | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham):
Is that a regression in Django 1.11? If so, it may be caused by
f734e2d4b2fc4391a4d097b80357724815c1d414 (#28170 was a similar issue). Can
you suggest a fix?
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:1>
Django
Reporter: bhargu | Owner: nobody
Type: Bug | Status: closed
Component: File | Version: 1.11
uploads/storage |
Keywords: selinux | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* status: new => closed
* resolution: => needsinfo
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:2>
Django
Reporter: bhargu | Owner: nobody
Type: Bug | Status: closed
Component: File | Version: 1.11
uploads/storage |
Severity: Normal | Resolution: needsinfo
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Phillip Marshall):
I'm also getting error code 13 after doing a large upgrade recently, but
am not using SELinux like bhargu. I have a CIFS mount set up in debian,
and am using a subclass of FileSystemStorage. Outside of python, the mount
is being finicky and throws permission errors when i try and chmod
anything in it, but adding removing and editing files works fine. In
previous django versions, I believe this kind of behavior was just
ignored. However my deployment environment has been upgraded as well, so
I'm not 100% sure of this.
Like OP says, this does not happen when the file size is smaller than
(DATA|FILE)_UPLOAD_MAX_MEMORY_SIZE (which default to 2.5 megs), and
setting those to a higher threshold works around the issue.
I wrote a simple patch below to allow for error 13, and this also remedied
the issue for me. I'm not sure if it's a good idea to merge, as code 13
seems to be a general permission error. But it would be nice to be able to
override this behaviour myself without having to do such a deep fork.
https://github.com/agrimgt/django/commit/f9b48086d083b1eee800ea752f2c3fd60a8cc448
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:3>
Django
Reporter: bhargu | Owner: nobody
Type: Bug | Status: new
uploads/storage |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Paolo Donadeo):
* status: closed => new
* resolution: needsinfo =>
Comment:
I have a question: why the ticket has been closed? I can confirm the issue
reported by bhargu still present in Django 2.1.3 and I can confirm that
the proposed patch:
https://github.com/agrimgt/django/commit/f9b48086d083b1eee800ea752f2c3fd60a8cc448
works.
This is a show stopper for us: Django simply don't work on a RHEL7 or
CentOS7 with SELinux enabled. To deploy Django 2.x I'm forced to fork.
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:4>
Django
--------------------------------------+------------------------------------
Reporter: bhargu | Owner: nobody
Type: Bug | Status: new
Component: File uploads/storage | Version: 1.11
Keywords: selinux | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* has_patch: 0 => 1
* needs_tests: 0 => 1
* stage: Unreviewed => Accepted
Comment:
I closed the issue because there wasn't a response to comment 1.
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:5>
Django
--------------------------------------+------------------------------------
Type: Bug | Status: new
Component: File uploads/storage | Version: 1.11
Keywords: selinux | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Paul Boddie):
One factor that may be involved is the user of the process when a Django
application is deployed using mod_wsgi.
I found that using the WSGIDaemonProcess directive with a user other than
apache (on Fedora, perhaps www-user on Debian), attempts to upload files
failed in the way described, but with the apache user involved (and with
appropriate permissions on directories for writing uploaded content) no
such error occurred.
It occurs to me that the apache user may be configured in SELinux to not
attempt to relabel files in an inappropriate way, or that the apache user
is able to preserve httpd_sys_rw_content_t labels whereas other users are
not. This is speculation, however, since I have not investigated this in
any depth (and the SELinux documentation is often incoherent or opaque).
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:6>
Django
--------------------------------------+------------------------------------
Type: Bug | Status: new
Component: File uploads/storage | Version: 1.11
Keywords: selinux | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Yuri Konotopov):
I created [https://github.com/django/django/pull/16056 PR 16056]
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:7>
Django
Reporter: bhargu | Owner: nobody
Type: Bug | Status: new
uploads/storage |
Severity: Normal | Resolution:
Keywords: selinux | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_tests: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:8>
Django
-------------------------------------+-------------------------------------
| Konotopov
Type: Bug | Status: assigned
Component: File | Version: 1.11
uploads/storage |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* owner: nobody => Yuri Konotopov
* needs_better_patch: 0 => 1
* status: new => assigned
* stage: Ready for checkin => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: bhargu | Owner: Yuri
| Konotopov
Type: Bug | Status: assigned
Component: File | Version: 1.11
uploads/storage |
Severity: Normal | Resolution:
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: bhargu | Owner: Yuri
| Konotopov
Component: File | Version: 1.11
uploads/storage |
Keywords: selinux | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"64e5ef1f17cd18cb8ca24f4e7107dfd28c18b378" 64e5ef1]:
{{{
#!CommitTicketReference repository=""
revision="64e5ef1f17cd18cb8ca24f4e7107dfd28c18b378"
Fixed #29027 -- Fixed file_move_safe() crash when moving files with
SELinux.
Thanks Florian Apolloner for the review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29027#comment:11>