Resource leaks from FieldFile's implicit open?

[Carsten Fuchs]

Dear group,

despite a lot of reading and searching I still don't understand how FieldFiles work.

The documentation at https://docs.djangoproject.com/en/4.0/ref/models/fields/#django.db.models.fields.files.FieldFile says:

> The API of FieldFile mirrors that of File, with one key difference: The object wrapped by the class is not necessarily a wrapper around Python’s built-in file object. Instead, it is a wrapper around *** the result of the Storage.open() method ***, which may be a File object, or it may be a custom storage’s implementation of the File API.

(I added markers to indicate the portion that I have trouble with.)

As far as I understand this, whenever a `FieldFile` is instantiated, `Storage.open()` is called. Thus, if the underlying storage is the `FileSystemStorage`, a Python file object is opened.

However, how is this file handle closed again?

It looks as if it stays open until garbage collected, that is, possibly for a very long time.

Another consequence besides the potential resource leak seems to be that the implicit open will raise an exception if the file that the `FieldFile` refers to doesn't exist. As I don't exactly understand when this opening occurs, a lot of code must be wrapped in try-except blocks to handle this?

Best regards,
Carsten

[Antonis Christofides]

I think that the documentation is not entirely clear. I wouldn't say that the object "wrapped by" FieldFile is "a wrapper around the result of" Storage.open(). It is actually FieldFile.open() that is a wrapper around Storage.open(). Therefore I think I'd rephrase as follows:

FieldFile is a subclass of File. While File is a wrapper around Python's built-in file object, FieldFile uses the underlying Storage.

As far as I can see by reading the code, you can write code like this:

with some_file_field:
    ...

Inside the "with" block you will probably access `some_file_field.file`. Apparently accessing that attribute is what opens the file (but accessing the `size` attribute also seems to be opening it, which is probably suboptimal.)

I'm not certain about files that don't exist. You might want to wrap the "with" block in a try-except, but I'm not certain about which exceptions you want to catch. It's probably ValueError and IOError.

Regards,

Antonis

Antonis Christofides
+30-6979924665 (mobile)

[Carsten Fuchs]

Hello,

many thanks for your reply!

Am 02.02.22 um 09:17 schrieb Antonis Christofides:

> It is actually FieldFile.open() that is a wrapper around Storage.open(). Therefore I think I'd rephrase as follows:
>
> FieldFile is a subclass of File. While File is a wrapper around Python's built-in file object, FieldFile uses the underlying Storage.

Yes, thanks, that makes a lot more sense than what's in the docs!

> Inside the "with" block you will probably access `some_file_field.file`. Apparently accessing that attribute is what opens the file (but accessing the `size` attribute also seems to be opening it, which is probably suboptimal.)

Yes, and e.g. the `width` attribute if it is an image...

I had a look at the code, but there are still a *lot* of open questions. At this time, I consider to not use FileField at all but to use plain CharFields instead to just store the file names and then to use the storage directly. This removes an entire layer of abstraction without which things seem to be a lot clearer.

Best regards,
Carsten