[Django] #36344: Accessing a related field after using defer() or only() unexpectedly overwrites unsaved state on model instances

[Django]

#36344: Accessing a related field after using defer() or only() unexpectedly
overwrites unsaved state on model instances
-------------------------------------+-------------------------------------
Reporter: Erick | Type:
| Uncategorized
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
When working with a model instance that was originally loaded with
`only()` or `defer()`, accessing any related object not cached on the
instance reverts unrelated fields to the value currently present in the
database.

This toy example reproduces the behavior:

{{{
class SomethingElse(Model):
...

class Something(Model):
foo = CharField(max_length=2)
bar = CharField(max_length=2)
other = ForeignKey(SomethingElse)


Something.objects.create(
foo="ab",
bar="cd",
other=SomethingElse.objects.create()
)

something = Something.objects.only("bar").first()
something.foo = "ef"
assert something.foo == "ef"

something.other # Accessing this deferred related field triggers the
behavior
assert something.foo == "ef" # AssertionError
}}}

This is surprising behavior, because application code may interact with
model instances without knowing how they were loaded from the database.
--
Ticket URL: <https://code.djangoproject.com/ticket/36344>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36344: Accessing a related field after using defer() or only() unexpectedly
overwrites unsaved state on model instances
-------------------------------------+-------------------------------------

Reporter: Erick | Owner: (none)
Type: Uncategorized | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution: invalid

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* resolution: => invalid
* status: new => closed

Comment:

I cannot reproduce against `main` or 5.2 (which you reported the issue
against) with

{{{#!diff
diff --git a/tests/defer/tests.py b/tests/defer/tests.py
index 989b5c63d7..1b38f2b5d3 100644
--- a/tests/defer/tests.py
+++ b/tests/defer/tests.py
@@ -184,6 +184,18 @@ def test_defer_fk_attname(self):
with self.assertNumQueries(1):
self.assertEqual(primary.related_id, self.p1.related_id)

+ def test_related_fetching(self):
+ Primary.objects.create(
+ name="foo",
+ value="bar",
+ related=Secondary.objects.create(first="first",
second="second"),
+ )
+ primary = Primary.objects.only("name").first()
+ primary.value = "ef"
+ self.assertEqual(primary.value, "ef")
+ primary.related
+ self.assertEqual(primary.value, "ef")
+

class BigChildDeferTests(AssertionMixin, TestCase):
@classmethod
}}}

At first I thought you ran into #35950 (which was fixed in 5.1.4) but I
can't reproduce against 5.1.3 either.

I'm afraid you might have omitted some details here.
--
Ticket URL: <https://code.djangoproject.com/ticket/36344#comment:1>

[Django]

#36344: Accessing a related field after using defer() or only() unexpectedly
overwrites unsaved state on model instances
-------------------------------------+-------------------------------------
Reporter: Erick | Owner: (none)
Type: Uncategorized | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Erick):

You're right, I over-simplified it. This appears to happen when:

1. An attribute is set on the model instance, and the name of the impacted
field ("value" in your example) is a substring of that attribute name;
2. The impacted field is accessed in the model instance constructor;

This repros on main:

{{{
diff --git a/tests/defer/models.py b/tests/defer/models.py
index 560e54c8c0..5b5f8fb86a 100644
--- a/tests/defer/models.py
+++ b/tests/defer/models.py
@@ -18,6 +18,9 @@ class Primary(models.Model):
def __str__(self):
return self.name

+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ self.value

class PrimaryOneToOne(models.Model):
name = models.CharField(max_length=50)
diff --git a/tests/defer/tests.py b/tests/defer/tests.py
index 989b5c63d7..11b7042a2f 100644
--- a/tests/defer/tests.py
+++ b/tests/defer/tests.py
@@ -360,3 +360,17 @@ class DeferredRelationTests(TestCase):
obj.second # Accessing a deferred field.
with self.assertNumQueries(0):
obj.primary_o2o
+
+

+ def test_related_fetching(self):
+ Primary.objects.create(
+ name="foo",
+ value="bar",
+ related=Secondary.objects.create(first="first",
second="second"),
+ )
+ primary = Primary.objects.only("name").first()

+ setattr(primary, "_any_value", "what")

+ primary.value = "ef"
+ self.assertEqual(primary.value, "ef")
+ primary.related
+ self.assertEqual(primary.value, "ef")

\ No newline at end of file
}}}


So it is a rather niche condition, admittedly. But still surprising.
--
Ticket URL: <https://code.djangoproject.com/ticket/36344#comment:2>

[Django]

#36344: Accessing a related field after using defer() or only() unexpectedly
overwrites unsaved state on model instances
-------------------------------------+-------------------------------------
Reporter: Erick | Owner: (none)
Type: Uncategorized | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

Accessing deferred fields in `__init__` is going to cause you a lot of
trouble #22858, #31435, #32660 and
[https://docs.djangoproject.com/en/5.2/ref/models/instances/#django.db.models.Model
is documented to be problematic].

It causes a silent N+1 query which explains why your value is overriden
when `value` is not part of the inclusion mask (accessing `self.value`
will cause it to be retrieved it from the database and then assigned on
the model instance).

If you ''really need'' to implement field retrieval in `__init__` you
should
[https://docs.djangoproject.com/en/5.2/ref/models/instances/#django.db.models.Model.get_deferred_fields
gate these retrieval behind get_deferred_fields].


{{{#!python
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
if "value" not in self.get_deferred_fields():
self.value
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36344#comment:3>

[Django]

#36344: Accessing a related field after using defer() or only() unexpectedly
overwrites unsaved state on model instances
-------------------------------------+-------------------------------------
Reporter: Erick | Owner: (none)
Type: Uncategorized | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Erick):

That makes sense, appreciate the explanation. I wonder if it would be
reasonable to update the docs to mention this bad behavior; as written, I
would have passed off the admonition about __init__ as probably not
related given that it just talks about infinite recursion.

> Additionally, referring to model fields within __init__ may potentially
result in infinite recursion errors in some circumstances.

Happy to contribute a PR if you think it's a good idea.
--
Ticket URL: <https://code.djangoproject.com/ticket/36344#comment:4>