[Django] #28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name

13 views
Skip to first unread message

Django

unread,
Oct 18, 2017, 3:00:13 PM10/18/17
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike | Owner: nobody
Hansen |
Type: | Status: new
Uncategorized |
Component: Database | Version: master
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Currently, `RelatedManager.get_prefetch_queryset` returns
`self.field.related_query_name()` as the `cache_name`. In the case where
no `related_name` has been set on the `ForeignKey`, then this does not
match with the `through_attr` used by `get_prefetcher`.

Using the models in `tests/prefetch_related/models.py`,

{{{#!python
BookWithYear.objects.prefetch_related('bookreview_set')
}}}
will use a `through_attr` of `bookreview_set`, but `bookreview` is what
will be placed in `_prefetched_objects_cache`.

I think `related_manager.field.remote_field.get_accessor_name()` should be
used instead.

I'll post a patch fixing this and adding a test case.

--
Ticket URL: <https://code.djangoproject.com/ticket/28723>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Oct 18, 2017, 3:16:00 PM10/18/17
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Uncategorized | Status: new
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mike Hansen):

* has_patch: 0 => 1


Comment:

A pull request for this can be found at
https://github.com/django/django/pull/9259

I wasn't sure on the best location for these tests so any guidance would
be helpful.

--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:1>

Django

unread,
Oct 18, 2017, 4:42:03 PM10/18/17
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Uncategorized | Status: new
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Mike Hansen:

Old description:

> Currently, `RelatedManager.get_prefetch_queryset` returns
> `self.field.related_query_name()` as the `cache_name`. In the case where
> no `related_name` has been set on the `ForeignKey`, then this does not
> match with the `through_attr` used by `get_prefetcher`.
>
> Using the models in `tests/prefetch_related/models.py`,
>
> {{{#!python
> BookWithYear.objects.prefetch_related('bookreview_set')
> }}}
> will use a `through_attr` of `bookreview_set`, but `bookreview` is what
> will be placed in `_prefetched_objects_cache`.
>
> I think `related_manager.field.remote_field.get_accessor_name()` should
> be used instead.
>
> I'll post a patch fixing this and adding a test case.

New description:

Currently, `RelatedManager.get_prefetch_queryset` returns
`self.field.related_query_name()` as the `cache_name`. In the case where
no `related_name` has been set on the `ForeignKey`, then this does not
match with the `through_attr` used by `get_prefetcher`.

Using the models in `tests/prefetch_related/models.py`,

{{{#!python
BookWithYear.objects.prefetch_related('bookreview_set')
}}}
will use a `through_attr` of `"bookreview_set"`, but `"bookreview"` is
what will be placed in `_prefetched_objects_cache`.

I think `related_manager.field.remote_field.get_accessor_name()` should be
used instead.

I'll post a patch fixing this and adding a test case.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:2>

Django

unread,
Oct 18, 2017, 6:14:08 PM10/18/17
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mike Hansen):

* type: Uncategorized => Bug


Old description:

> Currently, `RelatedManager.get_prefetch_queryset` returns
> `self.field.related_query_name()` as the `cache_name`. In the case where
> no `related_name` has been set on the `ForeignKey`, then this does not
> match with the `through_attr` used by `get_prefetcher`.
>
> Using the models in `tests/prefetch_related/models.py`,
>
> {{{#!python
> BookWithYear.objects.prefetch_related('bookreview_set')
> }}}
> will use a `through_attr` of `"bookreview_set"`, but `"bookreview"` is
> what will be placed in `_prefetched_objects_cache`.
>
> I think `related_manager.field.remote_field.get_accessor_name()` should
> be used instead.
>
> I'll post a patch fixing this and adding a test case.

New description:

Currently, `RelatedManager.get_prefetch_queryset` returns
`self.field.related_query_name()` as the `cache_name`. In the case where
no `related_name` has been set on the `ForeignKey`, then this does not
match with the `through_attr` used by `get_prefetcher`.

Using the models in `tests/prefetch_related/models.py`,

{{{#!python
BookWithYear.objects.prefetch_related('bookreview_set')
}}}
will use a `through_attr` of `"bookreview_set"`, but `"bookreview"` is
what will be placed in `_prefetched_objects_cache`.

I think `related_manager.field.remote_field.get_accessor_name()` should be
used instead.

https://github.com/django/django/pull/9259 is a pull request which fixes
this issue

--

--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:3>

Django

unread,
Nov 2, 2017, 10:16:15 AM11/2/17
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

* stage: Unreviewed => Ready for checkin


--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:4>

Django

unread,
Nov 2, 2017, 10:32:14 AM11/2/17
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Bug | Status: closed

Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham <timograham@…>):

* status: new => closed
* resolution: => fixed


Comment:

In [changeset:"514b2c989a948e3c59bda0da0c9427acf643cf5b" 514b2c98]:
{{{
#!CommitTicketReference repository=""
revision="514b2c989a948e3c59bda0da0c9427acf643cf5b"
Fixed #28723 -- Fixed RelatedManager's prefetch_related() cache name.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:5>

Django

unread,
Nov 2, 2017, 10:51:13 AM11/2/17
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mike Hansen):

Thanks Tim!

--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:6>

Django

unread,
Feb 3, 2018, 12:37:52 PM2/3/18
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Jon Dufresne):

* status: closed => new
* resolution: fixed =>
* stage: Ready for checkin => Accepted


Comment:

I believe this change is causing a regression as the
`_remove_prefetched_objects()` method was not updated to use the new cache
name. See line:

https://github.com/django/django/blob/514b2c989a948e3c59bda0da0c9427acf643cf5b/django/db/models/fields/related_descriptors.py#L578

I've created a [https://github.com/django/django/pull/9657 PR] to fix it.

--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:7>

Django

unread,
Feb 5, 2018, 11:03:51 AM2/5/18
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham <timograham@…>):

In [changeset:"8b21878357364a461bae711c4d0011a8f338b160" 8b218783]:
{{{
#!CommitTicketReference repository=""
revision="8b21878357364a461bae711c4d0011a8f338b160"
Refs #28723 -- Fixed stale prefetch_related cache after add/remove.

Regression in 514b2c989a948e3c59bda0da0c9427acf643cf5b.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:8>

Django

unread,
Feb 7, 2018, 3:29:44 PM2/7/18
to django-...@googlegroups.com
#28723: RelatedManager.get_prefetch_queryset returns "wrong" cache_name
-------------------------------------+-------------------------------------
Reporter: Mike Hansen | Owner: nobody
Type: Bug | Status: closed

Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

* status: new => closed
* resolution: => fixed


--
Ticket URL: <https://code.djangoproject.com/ticket/28723#comment:9>

Reply all
Reply to author
Forward
0 new messages