[Django] #36384: update_dimension_fields blocks getting model instances from the database when the external storage has issues

[Django]

#36384: update_dimension_fields blocks getting model instances from the database
when the external storage has issues
-------------------------------------+-------------------------------------
Reporter: Gerben Morsink | Type:
| Cleanup/optimization
Status: new | Component: Database
| layer (models, ORM)
Version: 5.1 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
It seems when a height_field or width_field are given, the
update_dimension_fields is always called on initialization of the model
instance. Thereby always doing a call to an external database if those
fields are empty.

This might seem obvious, but it means that when and external storage is
down, it is not possible anymore to get this Model instance anymore.

For example with S3 issue it might raise:

```
ClientError: An error occurred (403) when calling the HeadObject
operation: Forbidden
```

even when I simply want to get the object without the width and height,
like here:

```
UserAvatar.objects.get(user=user).only('id')
```

This is in my opinion an unexpected consequence of adding the height_field
and width_field to an ImageField.
What do you think, should width and height be lazily loaded somehow?
--
Ticket URL: <https://code.djangoproject.com/ticket/36384>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36384: update_dimension_fields blocks getting model instances from the database
when the external storage has issues
-------------------------------------+-------------------------------------

Reporter: Gerben Morsink | Owner: (none)
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 5.1
(models, ORM) | Resolution:
Severity: Normal | worksforme

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* resolution: => worksforme
* status: new => closed

Comment:

> even when I simply want to get the object without the width and height,
like here: `UserAvatar.objects.get(user=user).only('id')`

I cannot replicate this. Update dimension fields has logic to handle this,
and from my testing this works
{{{
# Nothing to update if the field doesn't have dimension fields or
if
# the field is deferred.
has_dimension_fields = self.width_field or self.height_field
if not has_dimension_fields or self.attname not in
instance.__dict__:
return
}}}

Can you provide a test or a way to replicate this behavior?
--
Ticket URL: <https://code.djangoproject.com/ticket/36384#comment:1>

[Django]

#36384: update_dimension_fields blocks getting model instances from the database
when the external storage has issues
-------------------------------------+-------------------------------------
Reporter: Gerben Morsink | Owner: (none)
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 5.1
(models, ORM) | Resolution:
Severity: Normal | worksforme
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

* easy: 1 => 0


Old description:

> It seems when a height_field or width_field are given, the
> update_dimension_fields is always called on initialization of the model
> instance. Thereby always doing a call to an external database if those
> fields are empty.
>
> This might seem obvious, but it means that when and external storage is
> down, it is not possible anymore to get this Model instance anymore.
>
> For example with S3 issue it might raise:
>
> ```
> ClientError: An error occurred (403) when calling the HeadObject
> operation: Forbidden
> ```
>

> even when I simply want to get the object without the width and height,
> like here:
>
> ```
> UserAvatar.objects.get(user=user).only('id')

> ```
>
> This is in my opinion an unexpected consequence of adding the
> height_field and width_field to an ImageField.
> What do you think, should width and height be lazily loaded somehow?

New description:

It seems when a height_field or width_field are given, the
update_dimension_fields is always called on initialization of the model
instance. Thereby always doing a call to an external database if those
fields are empty.

This might seem obvious, but it means that when and external storage is
down, it is not possible anymore to get this Model instance anymore.

For example with S3 issue it might raise:

`ClientError: An error occurred (403) when calling the HeadObject
operation: Forbidden`

even when I simply want to get the object without the width and height,
like here:

`UserAvatar.objects.get(user=user).only('id')`

This is in my opinion an unexpected consequence of adding the height_field
and width_field to an ImageField.
What do you think, should width and height be lazily loaded somehow?

--

Comment:

The quote code snippet is checking whether the `ImageField` has
height/width fields specified, so that's not the code that will raise an
exception. The report certainly seems plausible, even if it's unclear how
the situation could be improved. It might be the S3 storage backend rather
than Django that should be resilient to network failure.
--
Ticket URL: <https://code.djangoproject.com/ticket/36384#comment:2>