[Django] #23051: QuerySet.only() fail to work with reverse o2o relationships

Django

unread,

Jul 17, 2014, 6:12:12 AM7/17/14

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------
Reporter: vvd | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.6
(models, ORM) | Keywords: orm only reverse
Severity: Normal | relationship OneToOneField
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Specifying a field from reverse relationship model in the .only() queryset
method have no effect on compiled query:

{{{
# sample models
class Person(models.Model):
name = models.CharField(max_length=64)

class PersonExtra(models.Model):
bio = models.TextField()
information = models.TextField()
person = models.OneToOneField(Person)

# manage.py shell
>>> from testapp.models import Person
>>> print Person.objects.all().only('name').query
SELECT "testapp_person"."id", "testapp_person"."name" FROM
"testapp_person"
>>> print Person.objects.all().select_related('personextra').only('name',
'personextra__bio').query # expected table join and personextra__bio to be
loaded
SELECT "testapp_person"."id", "testapp_person"."name" FROM
"testapp_person"
}}}

defer() method works fine:

{{{
>>> print
Person.objects.all().select_related('personextra').defer('personextra__information').query
SELECT "testapp_person"."id", "testapp_person"."name",
"testapp_personextra"."id", "testapp_personextra"."bio",
"testapp_personextra"."person_id" FROM "testapp_person" LEFT OUTER JOIN
"testapp_personextra" ON ( "testapp_person"."id" =
"testapp_personextra"."person_id" )
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/23051>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Jul 19, 2014, 7:28:40 AM7/19/14

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------
Reporter: vvd | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.6

* needs_better_patch: => 0
* stage: Unreviewed => Accepted
* needs_tests: => 0
* needs_docs: => 0

Comment:

Hi,

I can indeed reproduce this.

Currently, `Person.objects.only('name', 'personextra__bio')` is accepted
by returns the wrong query. If supporting that use case is not possible,
it should at least raise an error instead of silently doing the wrong
thing.

Thanks.

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:1>

Django

unread,

Jul 26, 2014, 10:58:51 AM7/26/14

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------

Reporter: vvd | Owner: nip3o
Type: Bug | Status: assigned

Component: Database layer | Version: 1.6

Changes (by nip3o):

* status: new => assigned
* owner: nobody => nip3o

Comment:

Will try to at least make a test for this during EP14.

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:2>

Django

unread,

Jul 26, 2014, 11:19:18 AM7/26/14

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------

Reporter: vvd | Owner: nip3o
Type: Bug | Status: assigned

Component: Database layer | Version: 1.6

Comment (by nip3o):

Failing test:

{{{

diff --git a/tests/defer_regress/tests.py
b/tests/defer_regress/tests.py
index 63f9e97..5d70da2 100644
--- a/tests/defer_regress/tests.py
+++ b/tests/defer_regress/tests.py
@@ -144,6 +144,18 @@ class DeferRegressionTest(TestCase):
list(SimpleItem.objects.annotate(Count('feature')).only('name')),
list)

+ def test_ticket_23051(self):
+ item = Item.objects.create(value=1, name='item')
+ OneToOneItem.objects.create(item=item, name='one_to_one_item')
+
+ obj = Item.objects.only('name',
'one_to_one_item__name').get(name='item')
+
+ with self.assertNumQueries(0):
+ self.assertEqual(obj.name, 'item')
+
+ with self.assertNumQueries(0):
+ self.assertEqual(obj.one_to_one_item.name, 'one_to_one_item')
+
def test_only_and_defer_usage_on_proxy_models(self):
# Regression for #15790 - only() broken for proxy models
proxy = Proxy.objects.create(name="proxy", value=42)

}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:3>

Django

unread,

Jul 26, 2014, 12:51:10 PM7/26/14

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------
Reporter: vvd | Owner:

Type: Bug | Status: new
Component: Database layer | Version: master

Changes (by nip3o):

* status: assigned => new
* owner: nip3o =>
* version: 1.6 => master

Comment:

I looked in to this, and it seems that at least
`SQLCompiler.deferred_to_columns` (and all related methods called before
then) works as intended in this case. However, I will not be able to solve
this, even if I believe that it should not be unsolvable.

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:4>

Django

unread,

Feb 14, 2019, 12:25:47 AM2/14/19

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Dan Davis):

* cc: Dan Davis (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:5>

Django

unread,

Oct 26, 2019, 3:54:34 PM10/26/19

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------

Reporter: Vladimir Dmitriev | Owner: Paulo
Type: Bug | Status: assigned

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Paulo):

* owner: (none) => Paulo

* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:6>

Django

unread,

Oct 27, 2019, 12:11:44 PM10/27/19

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Paulo):

* has_patch: 0 => 1

Comment:

The original reported issue was fixed already, calling:

{{{

Person.objects.all().select_related('personextra').only('name',
'personextra__bio')
}}}

Gives the expected result (on 2.3 master branch).
But as pointed out in the comments above, calling:

{{{
# removed select_related()
Person.objects.all().only('name', 'personextra__bio')
}}}

Will silently fail to prefetch the personextra relationship.

I've opened a pr https://github.com/django/django/pull/11981 that raises a
ValueError when passing a reverse field to only(),
without prefetching the relationship first (as in the first example).

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:7>

Django

unread,

Dec 16, 2019, 10:52:45 AM12/16/19

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Baptiste Mispelon):

As noted in comment:7 above, the usecase is now supported when used in
conjunction with `select_related`.

I did some bisecting to track down the commit involved and found out that
two commits were actually involved:

1) With 3daa9d60be6672841ceed5bb812b6fb25950dc16, Django started raising
an error: `FieldError: Invalid field name(s) given in select_related:
'personextra'. Choices are: personextra`
2) With 0456a8b9e66608e2e7b7e64035d477f44c259c94, the error message
disappeared and the feature started working

For reference, here's the testcase that I used (using the models from the
original ticket):
{{{#!python
class TicketTestCase(TestCase):
def test_ticket(self):
p = Person.objects.create(name="Baptiste")
PersonExtra.objects.create(person=p, bio="yes")

qs =
list(Person.objects.select_related('personextra').only('name',
'personextra__bio'))

with self.assertNumQueries(0):
self.assertEqual(len(qs), 1)
self.assertEqual(qs[0].name, "Baptiste")
self.assertEqual(qs[0].personextra.bio, "yes")
}}}

I haven't looked in details at the proposed pull request, but I do have
some preliminary questions:

1) Is it fundamentally impossible to support using a reverse o2o field in
calls to `only()`? The PR adds an error message but I think it would be
better if we could fix the issue instead.
2) It seems that the problem exists for the case of a normal forward o2o
(`PersonExtra.objects.only('person__name')` in our example) and also for
reverse M2M (#30124). Does the PR address those too?

Thanks

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:8>

Django

unread,

Jan 15, 2020, 4:24:26 AM1/15/20

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by felixxm):

* needs_better_patch: 0 => 1

Comment:

Paulo, please check Baptiste comments and questions.

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:9>

Django

unread,

Jun 11, 2023, 10:07:23 PM6/11/23

to django-...@googlegroups.com

#23051: QuerySet.only() fail to work with reverse o2o relationships
-------------------------------------+-------------------------------------

Reporter: Vladimir Dmitriev | Owner: Paulo
Type: Bug | Status: assigned

Component: Database layer | Version: dev

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

FWIW #34612 was a related regression of 4.2 refactored deferred field
refactor. We still silence usage of `only` when it refers to related
models that are not selected so this ticket is still relevant though.

--
Ticket URL: <https://code.djangoproject.com/ticket/23051#comment:10>

Reply all

Reply to author

Forward