[Django] #33011: my_model.manyrelation_set.all() return empty value
Django
-------------------------------------+-------------------------------------
Reporter: Floréal | Owner: nobody
Cabanettes |
Type: Bug | Status: new
Component: Database | Version: 3.2
layer (models, ORM) |
Severity: Normal | Keywords: model set querying
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I have a problem which occurred since django 3.2 (no problem with django
3.1).
I use django rest framework. On the view, I have a model "Run" and a model
"Experiment". On the view, I get experiments with :
my_experiments = my_run.experiment_set.all()
But since django 3.2, this returns an empty queryset. If I add a
breakpoint in pycharm just after this line, the queryset is no more empty.
Also, this line works perferctly:
my_experiments = Experiment.objects.filter(run_id=my_run.id)
But I think the first code is more beautiful... so why this problem?
--
Ticket URL: <https://code.djangoproject.com/ticket/33011>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: model set querying | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham):
Hi, could you please provide minimal models and code to reproduce the
issue? Ideally, you could also
[https://docs.djangoproject.com/en/dev/internals/contributing/triaging-
tickets/#bisecting-a-regression bisect] to find the commit where the
behavior changed.
--
Ticket URL: <https://code.djangoproject.com/ticket/33011#comment:1>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Floréal Cabanettes):
Hi,
I found the commit responsible for this bug:
{{{
036f160733 ("Refs #20577 -- Deferred filtering of prefetched related
querysets by reverse m2o relation.", 2020-10-06)
}}}
The line which cause the problem is on the file
django/db/models/fields/related_descriptors.py, line 584 (581 at the
commit, 584 on tag 3.2.6) :
{{{
queryset._defer_next_filter = True
}}}
If I comment this line, the bug disappear.
My code is not public (is for my enterprise, not my choice). I try to
describe a similar architecture bellow:
I have 2 models, one is "Run", the other "Experiment", related by a
OneToMany relationship:
{{{
class Run(model.Model):
label = models.CharField(max_length=100, null=False, blank=False)
owner = models.ForeignKey(User, null=True, blank=True,
on_delete=models.PROTECT, related_name="run_owner")
class Experiment(model.Model):
run = models.ForeignKey(Run, null=False, on_delete=models.PROTECT)
index1 = models.ForeignKey(Index, null=False,
on_delete=models.PROTECT, related_name="index1")
index2 = models.ForeignKey(Index, null=True, blank=True,
on_delete=models.PROTECT, related_name="index2")
}}}
I use the django rest framework to call the server through REST API. For
one endpoint, I get a run, and I try to get it's experiments, to then make
a copy of them:
{{{
@action(detail=True, methods=["post"])
def copy(self, request, pk=None):
try:
my_run = self.get_queryset().get(pk=pk)
except ObjectDoesNotExist:
return Response({"detail": _("Run with id '%s' does not exists") %
pk},
status=status.HTTP_404_NOT_FOUND)
my_experiments = my_run.experiment_set.all()
for my_experiment in my_experiments:
my_experiment.pk = None
...
my_experiment.save()
}}}
and the problem is that `my_experiments` is empty with django 3.2.6 (and
no more empty with the line commented like said before).
I hope this will help to solve the problem...
--
Ticket URL: <https://code.djangoproject.com/ticket/33011#comment:2>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Carlton Gibson):
Hi Floréal — thanks for the further detail.
Can you give some indication of the prefetch usage in the `get_queryset()`
method here please:
{{{
my_run = self.get_queryset().get(pk=pk)
}}}
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/33011#comment:3>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Floréal Cabanettes):
This apply some filters on the data (like filters given in input).
But it's not linked to this problem. I try to replace this line by:
{{{
my_run = self.queryset.get(pk=pk)
}}}
or by :
{{{
my_run = Run.objects.get(pk=pk)
}}}
and the problem is still the same.
Replying to [comment:3 Carlton Gibson]:
> Hi Floréal — thanks for the further detail.
>
> Can you give some indication of the prefetch usage in the
`get_queryset()` method here please:
>
> {{{
> my_run = self.get_queryset().get(pk=pk)
> }}}
>
>
> Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/33011#comment:4>
Django
-------------------------------------+-------------------------------------
Component: Database layer | Version: 3.2
(models, ORM) |
Keywords: model set querying | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => needsinfo
Comment:
Hi Floréal.
I can't see how to reproduce this with the info given. (There must be
something else going on in your project.)
Django's `model_fields` tests use these models:
{{{
class Foo(models.Model):
a = models.CharField(max_length=10)
d = models.DecimalField(max_digits=5, decimal_places=3)
def get_foo():
return Foo.objects.get(id=1).pk
class Bar(models.Model):
b = models.CharField(max_length=10)
a = models.ForeignKey(Foo, models.CASCADE, default=get_foo,
related_name='bars')
}}}
This is equivalent to yours here I think (in the relevant respects).
Adding this test passes:
{{{
diff --git a/tests/model_fields/test_foreignkey.py
b/tests/model_fields/test_foreignkey.py
index 0cd6d62a55..c51b6d978a 100644
--- a/tests/model_fields/test_foreignkey.py
+++ b/tests/model_fields/test_foreignkey.py
@@ -12,6 +12,14 @@ from .models import Bar, FkToChar, Foo,
PrimaryKeyCharModel
class ForeignKeyTests(TestCase):
+ def test_reverse_relation(self):
+ a = Foo.objects.create(id=1, a='abc', d=Decimal('12.34'))
+ # Bar FK defaults to Foo with id=1
+ b = Bar.objects.create(b='bcd')
+ c = Bar.objects.create(b='bcd')
+ d = Bar.objects.create(b='bcd')
+ self.assertSequenceEqual(a.bars.all(), [b, c, d])
+
def test_callable_default(self):
"""A lazy callable may be used for ForeignKey.default."""
a = Foo.objects.create(id=1, a='abc', d=Decimal('12.34'))
}}}
This is already covered elsewhere in the test suite (many times).
Can I ask you to provide a minimal reproduce of your issue?
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/33011#comment:5>