[Django] #34674: Updating the file contents of a Django FileField during upload results in I/O error
Django
I/O error
------------------------------------------------+------------------------
Reporter: Jeroen Jacobs | Owner: nobody
Type: Bug | Status: new
Component: File uploads/storage | Version: 4.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+------------------------
I'm trying to encrypt the contents of a file that is being uploaded. This
is the relevant code snippet:
{{{
class AppFile(models.Model):
app_file = models.FileField(upload_to=upload_to,
validators=[validate_file_size])
encrypted_data_key = models.CharField(max_length=500, blank=True)
def encrypt_file_with_data_key(self, data_key):
cipher = Fernet(data_key)
with self.app_file.open(mode='rb') as file:
file_data = file.read()
encrypted_data = cipher.encrypt(file_data)
with self.app_file.open(mode='wb') as encrypted_file:
encrypted_file.write(encrypted_data)
def save(self, *args, **kwargs):
if self._state.adding is True:
# New image being uploaded
encrypted_data_key, data_key =
self.generate_data_key_from_vault()
self.encrypted_data_key = encrypted_data_key
# Encrypt the uploaded image file
self.encrypt_file_with_data_key(data_key)
super().save(args, kwargs)
}}}
I prefer this approach as this is agnostic of the StorageProvider being
used. I also want to avoid detaching the file to a temporary folder, and
re-attach it after encryption.
However, this results in the following error:
{{{
Traceback (most recent call last):
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/views/generic/base.py", line 84, in view
return self.dispatch(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/views/generic/base.py", line 119, in dispatch
return handler(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/views/generic/edit.py", line 184, in post
return super().post(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/views/generic/edit.py", line 153, in post
return self.form_valid(form)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/contrib/messages/views.py", line 12, in form_valid
response = super().form_valid(form)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/views/generic/edit.py", line 135, in form_valid
self.object = form.save()
^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/forms/models.py", line 548, in save
self.instance.save()
File "/Users/jeroenjacobs/PycharmProjects/myapp/mainapp/models.py", line
90, in save
self.encrypt_file_with_data_key(data_key)
File "/Users/jeroenjacobs/PycharmProjects/myapp/mainapp/models.py", line
77, in encrypt_file_with_data_key
with self.app_file.open(mode='wb') as encrypted_file:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/db/models/fields/files.py", line 80, in open
self.file.open(mode)
File "/Users/jeroenjacobs/.pyenv/versions/myapp/lib/python3.11/site-
packages/django/core/files/uploadedfile.py", line 115, in open
self.file.seek(0)
ValueError: I/O operation on closed file.
}}}
Reading the contents doesn't seem to be a problem, but writing seems to
generate an error, despite `open` being called.
--
Ticket URL: <https://code.djangoproject.com/ticket/34674>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
I/O error
Reporter: Jeroen Jacobs | Owner: nobody
Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Natalia Bidart):
* status: new => closed
* resolution: => invalid
Comment:
Hello,
This report seems better suited to be a support request. The best place to
get answers to your issue is using any of the user support channels from
[https://docs.djangoproject.com/en/dev/faq/help/#how-do-i-do-x-why-
doesn-t-y-work-where-can-i-go-to-get-help this link].
Since the goal of this issue tracker is to track issues about Django
itself, and your issue seems, at first, be located in your custom code,
I'll be closing this ticket as invalid.
If, after debugging, you find out that this is indeed a bug in Django,
please re-open with the specific details.
Thank you!
--
Ticket URL: <https://code.djangoproject.com/ticket/34674#comment:1>
Django
I/O error
-------------------------------------+-------------------------------------
Reporter: Jeroen Jacobs | Owner: nobody
Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: invalid =>
Comment:
I am reopening this ticket, as is this is clearly a bug and not a support
request.
The error is not in my custom code! The error occurs when writing to a
Django FileField which is opened for writing. This not custom code, I'm
calling core Django code here.
Please take a good look at the code:
{{{
with self.app_file.open(mode='wb') as encrypted_file:
encrypted_file.write(encrypted_data)
}}}
app_file is a `FileField` and FileField is pure Django class, not custom
code!
Please read the description again: I'm writing to a Django FileField which
was successfully opened, and I'm getting an error that the file is closed!
This is clearly a bug! Did you even try to reproduce it using my example
code?
--
Ticket URL: <https://code.djangoproject.com/ticket/34674#comment:2>
Django
I/O error
-------------------------------------+-------------------------------------
Reporter: Jeroen Jacobs | Owner: nobody
Type: Bug | Status: new
Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham):
Hi Jeroen, to confirm it's a bug, you should explain where Django is at
fault.
--
Ticket URL: <https://code.djangoproject.com/ticket/34674#comment:3>
Django
I/O error
-------------------------------------+-------------------------------------
Reporter: Jeroen Jacobs | Owner: nobody
Type: Bug | Status: new
Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Jeroen Jacobs):
Getting an error that a file is closed when writing to it, when it was
successfully opened for writing seems like a fault to me, no?
This is not a standard file object, but a Django file object if I
understand the docs correctly
(https://docs.djangoproject.com/en/4.2/ref/files/file/). It's certainly
not a file permission issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/34674#comment:4>
Django
I/O error
-------------------------------------+-------------------------------------
Reporter: Jeroen Jacobs | Owner: nobody
Type: Bug | Status: new
Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Jeroen Jacobs):
In the unlikely case it's not a bug, it's certainly undefined behaviour
and maybe this should be documented somewhere.
So far, I've heard ZERO explanation why the reading the contents of newly
uploaded file works without a problem, but writing gives this bizar error.
--
Ticket URL: <https://code.djangoproject.com/ticket/34674#comment:5>
Django
I/O error
-------------------------------------+-------------------------------------
Reporter: Jeroen Jacobs | Owner: nobody
Component: File | Version: 4.2
uploads/storage |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
Hello Jeroen,
Please don't re-open tickets closed by Django maintainers/fellows. The
code you shared, while indeed uses the Django provided `FileField`, does
make a custom/non traditional use of it. Please reach out to the user
support channels as suggested, where you can get help on whether the
approach you took to encrypt the file content is sound or if there are
other considerations that are needed.
I did read the ticket description, I understand you may feel frustrated
but please follow the [https://www.djangoproject.com/conduct/ Django Code
of Conduct], and please see
[https://docs.djangoproject.com/en/dev/internals/contributing/bugs-and-
features/#reporting-bugs the guidelines on reporting bugs].
The example code you provided is incomplete and does not work. There are
undefined names and missing imports, and the stacktrace shows that a view
and a form are involved which you did not provide. There is also lacking
details of the settings of the project, such as the file storage backend
that is being used.
As recommended, please reach out for help in the user forum or Discord
channel, where there may be a bigger audience able to help in getting to
the root of the issue. If this is indeed a Django bug, please reopen with
a small yet complete Django project with the necessary models and other
files to reproduce (I'd advice leaving encryption out of the code to
reduce the scope of the reproducer).
--
Ticket URL: <https://code.djangoproject.com/ticket/34674#comment:6>
Django
I/O error
-------------------------------------+-------------------------------------
Reporter: Jeroen Jacobs | Owner: nobody
Type: Bug | Status: closed
Component: File | Version: 4.2
uploads/storage |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* resolution: invalid => duplicate
Comment:
Duplicate of #33023.
--
Ticket URL: <https://code.djangoproject.com/ticket/34674#comment:7>