[Django] #33651: Support prefetch GenericForeignKey with custom queryset.
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: nobody
Type: New | Status: new
feature |
Component: Database | Version: 4.0
layer (models, ORM) |
Severity: Normal | Keywords: GenericForeignKey
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
For example:
{{{
class Node(models.Model):
content_type = models.ForeignKey(ContentType,
on_delete=models.CASCADE)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey("content_type", "object_id")
class AbstractNodeItem(models.Model):
name = models.CharField(max_length=100)
class ItemA(AbstractNodeItem):
a_content = models.TextField()
class ItemB(AbstractNodeItem):
b_content = models.TextField()
}}}
Currently we can't list all node with only `content_object.name`
prefetched.
--
Ticket URL: <https://code.djangoproject.com/ticket/33651>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: nobody
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
If I understand correctly you'd like to be able to have the prefetch
querysets issued when doing
`Node.objects.prefetch_related('content_object')` for `ItemA` and `ItemB`
be `.objects.only('name')`.
I've had to implement such a feature in the past and ended up with writing
[https://gist.github.com/charettes/3dcdec3bf66257b0299455a70559f47d my
own] `GenericForeignKeySubclass` that would allow for the following
{{{#!python
class Node(models.Model):
content_type = models.ForeignKey(ContentType,
on_delete=models.CASCADE)
object_id = models.PositiveIntegerField()
content_object = PrefetchedManagerGenericForeignKey("name_only",
"content_type", "object_id")
class AbstractNodeItem(models.Model):
name = models.CharField(max_length=100)
class NameOnlyManager(models.Manager):
def get_queryset(self):
return super().only("name")
class ItemA(AbstractNodeItem):
a_content = models.TextField()
name_only = NameOnlyManager()
class ItemB(AbstractNodeItem):
b_content = models.TextField()
name_only = NameOnlyManager()
}}}
If we were to do a proper implementation here though I think that a
`GenericPrefetch(Prefetch)` subclass would make for a better API as that's
the current way custom querysets are provided to the prefetching logic.
Something along
{{{#!python
Node.objects.prefetch_related(
GenericPrefetch(
"content_object", querysets={
ItemA: ItemA.objects.only("a"),
ItemB: ItemB.objects.only("a"),
}
)
)
}}}
Accepting on the basis that I think this would be a valuable feature.
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:1>
Django
Reporter: elonzh | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette):
* component: Database layer (models, ORM) => contrib.contenttypes
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:2>
Django
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Gullapalli Saisurya Revanth):
* owner: nobody => Gullapalli Saisurya Revanth
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Gullapalli Saisurya Revanth):
* has_patch: 0 => 1
Comment:
PR -> [https://github.com/django/django/pull/15636 #15636]
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_docs: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_docs: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_docs: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_docs: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: David Wobrock (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
#24272 was a duplicate. One thing I noticed while reviewing it is that the
signature of `GenericPrefetch` would likely be ergonomic as
`GenericPrefetch(relation: str, querysets: list[QuerySet])` instead of
`querysets: dict[Model, QuerySet]` since the model can be inferred from
`queryset.model` anyway.
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Todor Velichkov (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Gullapalli Saisurya Revanth):
Yes Simon. The change from querysets from dictionary to list is also
suggested by Mariusz Felisiak. I have recently updated the PR with this
change.
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:16>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
Comment:
Per Simon's comment.
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:17>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Rohan Nahata (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:18>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Todor Velichkov):
Hi everyone, I took a brief look at the conversation in the pull request
[https://github.com/django/django/pull/15636 #15636] and I have a
question about the issue being discussed there (passing list of querysets
into an queryset argument).
Is there anything specific about this, that is enforcing the use of a list
of querysets? Is it possible to send as many `GenericPrefetch` objects as
the number of querysets being required ? For example:
{{{
Node.objects.prefetch_related(
GenericPrefetch("content_object", queryset=ItemA.objects.all()),
GenericPrefetch("content_object", queryset=ItemB.objects.all()),
)
}}}
this is in the spirit of the current `Prefetch` class, which actually
allows you to prefetch multiple levels of objects, splitted into different
prefetch calls. i.e:
{{{
Book.objects.prefetch_related(
Prefetch("author", queryset=Author.objects.all()),
Prefetch("author__house", queryset=House.objects.all()),
)
}}}
And now I'm a little bit repeating myself with ticket [ticket:24272], but
this could probably open the door for reusing `related_query_name` when
there is a defined `GenericRelation` which could give us the following
interface w/o using the generic `content_object` argument:
{{{
TaggedItem.objects.all().prefetch_related(
GenericPrefetch('books',
queryset=Book.objects.all().select_related('author')),
GenericPrefetch('movies',
queryset=Movie.objects.all().select_related('director')),
)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:19>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by revanthgss):
Hi,
I think there is no way to add two lookups on same foreign key on same
model as currently we populate the prefetch cache for this foreign key.
Anyway, for a generic foreign key currently giving a queryset is not
supported. To add this support I think the best way is to have a class
that extends the prefetch class which handles all the custom logic of
prefetching which is what we are working on in this issue.
To implement what you said i.e adding two Prefetch of content object in
prefetch_related function. Some of the logic will need to be handled in
the prefetch_related function which is not good.
Replying to [comment:19 Todor Velichkov]:
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:20>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: joeli (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:21>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner:
| Gullapalli Saisurya Revanth
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Clément Escolano (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:22>
Django
-------------------------------------+-------------------------------------
| Escolano
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Clément Escolano):
* owner: Gullapalli Saisurya Revanth => Clément Escolano
Comment:
[https://github.com/django/django/pull/17136 Link to the pull request]
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:23>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: Clément
| Escolano
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:24>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: Clément
| Escolano
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
* needs_tests: 0 => 1
* needs_docs: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:25>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: Clément
| Escolano
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
* needs_tests: 1 => 0
* needs_docs: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:26>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: Clément
| Escolano
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
Keywords: GenericForeignKey | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:27>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: Clément
| Escolano
Type: New feature | Status: assigned
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:28>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: Clément
| Escolano
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution: fixed
Keywords: GenericForeignKey | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"cac94dd8aa2fb49cd2e06b5b37cf039257284bb0" cac94dd8]:
{{{
#!CommitTicketReference repository=""
revision="cac94dd8aa2fb49cd2e06b5b37cf039257284bb0"
Fixed #33651 -- Added support for prefetching GenericForeignKey.
Co-authored-by: revanthgss <revan...@almabase.com>
Co-authored-by: Mariusz Felisiak <felisiak...@gmail.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:29>
Django
-------------------------------------+-------------------------------------
Reporter: elonzh | Owner: Clément
| Escolano
Type: New feature | Status: closed
Component: | Version: 4.0
contrib.contenttypes |
Severity: Normal | Resolution: fixed
Keywords: GenericForeignKey | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
In [changeset:"817bc5800b40bcc74534de5e5176919cb826494f" 817bc580]:
{{{#!CommitTicketReference repository=""
revision="817bc5800b40bcc74534de5e5176919cb826494f"
Refs #33651 -- Removed Prefetch.get_current_queryset() and
get_prefetch_queryset() per deprecation timeline.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33651#comment:30>