[Django] #29789: Support nested relations in FilteredRelation's condition
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Riccardi |
Type: New | Status: new
feature |
Component: Database | Version: 2.0
layer (models, ORM) | Keywords: FilteredRelation
Severity: Normal | nested relations
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
As the documentation explains, FilteredRelation's condition do not support
nested relations:
{{{
>>> Restaurant.objects.annotate(
... pizzas_with_toppings_startswith_n=FilteredRelation(
... 'pizzas__toppings',
... condition=Q(pizzas__toppings__name__startswith='n'),
... ),
... )
Traceback (most recent call last):
...
ValueError: FilteredRelation's condition doesn't support nested
relations (got 'pizzas__toppings__name__startswith').
}}}
It would be great to support nested relations in FilteredRelation's
condition (I encountered this limitation multiple times recently).
--
Ticket URL: <https://code.djangoproject.com/ticket/29789>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: New feature | Status: new
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage:
nested relations | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Nicolas Delaby (added)
Comment:
Nicolas, do you remember the reason for the restriction?
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:1>
Django
-------------------------------------+-------------------------------------
Type: New feature | Status: new
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage:
nested relations | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Nicolas Delaby):
I simply couldn't find a way to make it work, I don't remember exactly why
and I didn't have real incitement to do it at that time.
The explicit ValueError is more a gate keeper preventing users to follow a
path we know will fail cryptically.
My suggestion would be start writing some tests, removing the ValueError
and see how it goes.
My priority at that time was to land the feature even with partial
support. Now that a big chunk is there, we can more easily improve the
support so nested relations is supported.
Unfortunately I'm at the moment away from django stack, so I Invite you,
Thomas , to give it a try, if it is important for you. You might go
further than I could.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:2>
Django
-------------------------------------+-------------------------------------
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
nested relations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* version: 2.0 => master
* stage: Unreviewed => Accepted
Comment:
If I remember correctly there was some weirdness when dealing with
exclusion of such nested filtered relations.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:3>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Noumbissi Valere Gille Geovan
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:4>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by saber solooki):
Are you support this feature in next version or not? thank's in advance
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:5>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
#30349 addressed an issue when using `exclude` with `FilteredRelation`
that might explain the `ValueError` raised when trying to add support for
nested relations.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:6>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Reupen Shah (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:7>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Alex Scott (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:8>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Alex Scott):
Noumbissi, just wondering if you're working on this? And if so, how it's
progressing?
This problem (of needing to join to a subquery or add additional
conditions to a join that is nested more than once) seems to come up quite
a bit. I've posted my own question on StackOverflow:
https://stackoverflow.com/questions/58895627/django-queryset-with-
aggregation-and-filtered-left-join-multiple-levels-deep
In the absence of this feature, it'd be nice to have someone intimately
familiar with the ORM propose how they would tackle this as I haven't seen
a "this feature is not yet available, but as a workaround, try this
instead..". Would be very helpful.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:9>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by ferrants):
Noumbissi, Alex Scott, Nicolas Delaby
I've opened a PR for supporting Nested FilteredRelations in 2.2
I can also support this for 3.0, but I made the PR for 2.2 because I'm
using 2.2. Happy to open the PR for 3.0 as well.
Feedback is appreciated, I included comments in other places I had
examined for making the change.
https://github.com/django/django/pull/12288
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:10>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by felixxm):
Matt, we don't accept new features to the Django 2.2 and 3.0. Please
create a PR against master branch.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:11>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matt Ferrante):
Thanks for the heads-up felixxm. Opened one against master:
https://github.com/django/django/pull/12293
I'll work through the test failures.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:12>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matt Ferrante):
Got the tests passing in https://github.com/django/django/pull/12293
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:13>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/12333 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:14>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:15>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matt Ferrante):
felixxm, I've updated the PR, can you take another look? Thanks!
https://github.com/django/django/pull/12333
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:16>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by felixxm):
Matt, Thanks, you don't need to ping me, if PR is ready for another review
just uncheck ''"Patch needs improvement"''.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:17>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:18>
Django
-------------------------------------+-------------------------------------
| Valere Gille Geovan
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matt Ferrante):
Patch available here, rebased and it's good to go!
https://github.com/django/django/pull/12333
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:19>
Django
-------------------------------------+-------------------------------------
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: Noumbissi Valere Gille Geovan => (none)
* status: assigned => new
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:20>
Django
-------------------------------------+-------------------------------------
| Ferrante
Type: New feature | Status: assigned
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: (none) => Matt Ferrante
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:21>
Django
-------------------------------------+-------------------------------------
| Ferrante
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matt Ferrante):
felixxm, should I be doing something else to get this another review?
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:22>
Django
-------------------------------------+-------------------------------------
| Ferrante
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by felixxm):
Matt, you can ask any friend to review it. I'm sorry but you must be
patient we have many PRs in a review queue.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:23>
Django
-------------------------------------+-------------------------------------
| Ferrante
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matt Ferrante):
I've been using a branch with this patch for nested FilteredRelations in
production for 4 months without issues. It works really well and added a
lot of flexibility and allowed for performance optimizations which helped
with our analytical query speed. Hoping this gets approved and changes to
"Ready for checkin" soon!
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:24>
Django
-------------------------------------+-------------------------------------
| Ferrante
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
nested relations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
I think the proposed patch is ready to be merged. Looks like current
implementation of `FilteredRelation(condition=~Q)` might do weird things
under some circumstances but that seems like an artifact of how exclusions
against multi-valued relationship is currently implemented.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:25>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas Riccardi | Owner: Matt
| Ferrante
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
nested relations | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:26>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas Riccardi | Owner: Matt
| Ferrante
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Ready for
nested relations | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by felixxm):
[https://github.com/django/django/pull/13170 PR with final edits]
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:27>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas Riccardi | Owner: Matt
| Ferrante
Component: Database layer | Version: master
(models, ORM) |
Keywords: FilteredRelation | Triage Stage: Ready for
nested relations | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"7d6916e82700896ef23c50720d8cf0de4d6b9074" 7d6916e]:
{{{
#!CommitTicketReference repository=""
revision="7d6916e82700896ef23c50720d8cf0de4d6b9074"
Fixed #29789 -- Added support for nested relations to FilteredRelation.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:28>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas Riccardi | Owner: Matt
| Ferrante
Type: New feature | Status: closed
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: FilteredRelation | Triage Stage: Ready for
nested relations | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"e4a5527d1dc2f8183883931560f3a6dcdef0ab0c" e4a5527d]:
{{{
#!CommitTicketReference repository=""
revision="e4a5527d1dc2f8183883931560f3a6dcdef0ab0c"
Refs #29789 -- Added more tests for FilteredRelation with condition
outside of relation name.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:29>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas Riccardi | Owner: Matt
| Ferrante
(models, ORM) |
Severity: Normal | Resolution:
nested relations | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* resolution: fixed =>
* status: closed => new
Comment:
This bug resurufeced wiht 5.2. (after upgrading from 5.1.9). The queryset
causing this issue ("ValueError: FilteredRelation's condition doesn't
support nested relations deeper than the relation_name (got
'conflict_of_interests__panellist__user' for 'conflict_of_interests')."):
{{{
queryset = queryset.filter(round__panellists__user=user,
state="submitted").annotate(
coi=FilteredRelation(
"conflict_of_interests",
condition=Q(conflict_of_interests__panellist__user=user),
)
)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:30>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas Riccardi | Owner: Matt
| Ferrante
(models, ORM) |
nested relations | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* resolution: => fixed
* status: new => closed
Comment:
5.2 fixed a case where the check raising `ValueError` was being evaded for
implicit `__exact` lookups. If your queryset raises an error on 5.1.9 when
adding `__exact` after `user`, then this is not a bug. See #36029.
When reporting a bug, please open a new issue: you can always reference
other tickets while doing so. Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/29789#comment:31>