[Django] #32413: File upload permission denied error on large files
Django
-----------------------------------------+------------------------
Reporter: James Miller | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
Hi, if an uploaded file exceeds the DATA_UPLOAD_MAX_MEMORY_SIZE of 2.5 mb,
django starts streaming the file data to a file in /tmp.
When finished it copies it over to the its required file path location,
but despite the permissions being set correctly, a permission denied
error occurs.
{{{
Request Method: POST
Request URL: http://127.0.0.1:8000/xxxx/upload/
Django Version: 3.1.5
Exception Type: PermissionError
Exception Value:
[Errno 13] Permission denied:
'/opt/django_forum/django_forum/media/uploads/users/xxxxx
/Albert_Memorial_London_-_May_2008_vWcDQT6.jpg'
Exception Location: /usr/local/lib/python3.9/shutil.py, line 329, in
_copyxattr
Python Executable: /usr/local/bin/python
Python Version: 3.9.1
Python Path:
['/opt/django_forum/django_forum',
'/opt/django_forum/django_forum',
'/etc/opt/django_forum',
'/opt/django_forum',
'/usr/local/lib/python39.zip',
'/usr/local/lib/python3.9',
'/usr/local/lib/python3.9/lib-dynload',
'/usr/local/lib/python3.9/site-packages']
}}}
I have tried setting the following settings:
{{{
FILE_UPLOAD_DIRECTORY_PERMISSIONS = 0o755
FILE_UPLOAD_PERMISSIONS = 0o644
}}}
Most disturbingly, the files are created in their final position with 0755
permissions.
I am using a containerised workflow as opposed to a venv but I am not
certain that that should cause any issues.
--
Ticket URL: <https://code.djangoproject.com/ticket/32413>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: James Miller | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by James Miller:
Old description:
New description:
Most disturbingly, the large files are created in their final position
with 0755 permissions.
I am using a containerised workflow as opposed to a venv but I am not
certain that that should cause any issues.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/32413#comment:1>
Django
Reporter: James Miller | Owner: nobody
Component: File | Version: 3.1
uploads/storage |
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* status: new => closed
* resolution: => needsinfo
* component: Uncategorized => File uploads/storage
* type: Uncategorized => Bug
Comment:
I don't think there are enough details here to confirm that Django is at
fault. Please debug the issue and if Django is at fault, reopen with
details. Thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/32413#comment:2>
Django
Reporter: James Miller | Owner: nobody
Component: File | Version: 3.1
uploads/storage |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by René Fleschenberg):
* status: closed => new
* resolution: needsinfo =>
Comment:
We got some more information on IRC. Django calls copystat() at
https://github.com/django/django/blob/master/django/core/files/move.py#L71
which, in this specific case, raises PermissionError with EACCES (13).
Django catches the exception, but reraises it unless errno is EPERM:
https://github.com/django/django/blob/master/django/core/files/move.py#L76
I don't know what exactly causes the PermissionError here, or if it would
be good to also swallow EACCES. James mentioned that SELinux might play a
role here.
Looks like this needs more research, but I think we should keep the ticket
open for now. Since the permissions end up being less restrictive than the
FILE_UPLOAD_PERMISSIONS setting, this also has security implications.
--
Ticket URL: <https://code.djangoproject.com/ticket/32413#comment:3>
Django
Reporter: James Miller | Owner: nobody
Component: File | Version: 3.1
uploads/storage |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by René Fleschenberg):
* cc: René Fleschenberg (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/32413#comment:4>
Django
Reporter: James Miller | Owner: nobody
Component: File | Version: 3.1
uploads/storage |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by James Miller):
Ok, so I am investigating selinux contexts on files and directories in
containers, and it seems I may have volume mounted the directories that I
share with the host using incorrect selinux context flags. I used a
lower case z (share context between containers) when I should have used an
upper case Z (context is private to container).
I am going to recreate my container with the correct selinux flag on
volume mounts and see if it fixes the error.
--
Ticket URL: <https://code.djangoproject.com/ticket/32413#comment:5>
Django
Reporter: James Miller | Owner: nobody
Component: File | Version: 3.1
uploads/storage |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by James Miller):
So I recreated a pod using the more correct selinux context switch, but
with no luck regarding this issue.
I think it is probably an issue with python itself. see:
https://bugs.python.org/issue38893
--
Ticket URL: <https://code.djangoproject.com/ticket/32413#comment:6>
Django
Reporter: James Miller | Owner: nobody
Component: File | Version: 3.1
uploads/storage |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* status: new => closed
* resolution: => invalid
Comment:
Closing as invalid, unless someone can prove it's Django's fault or there
is anything we can improve in Django itself.
--
Ticket URL: <https://code.djangoproject.com/ticket/32413#comment:7>