[Django] #36682: RecursionError when traversing several reverse relations with FilteredRelation

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------
Reporter: REGNIER Guillaume | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Normal
Keywords: FilteredRelation | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
When declaring a filtered relation with a condition on the 2-distant
reverse relation, query compilation fails with a RecursionError

=== Steps to Reproduce:
{{{#!python
class Company(models.Model):
pass

class Employee(models.Model):
company = models.ForeignKey(Company, on_delete=models.CASCADE,
related_name="employees")

class Mission(models.Model):
employee = models.ForeignKey(Employee, on_delete=models.CASCADE,
related_name="missions")
name = models.TextField()
deadline = models.DateTimeField()
}}}

{{{#!python
qs = Company.objects.alias(
specific_mission=FilteredRelation(
"employees__missions",
condition=Q(employees__missions__name="specific"),
)
)
qs = qs.filter(specific_mission__deadline__lt=localtime())
list(qs)
}}}

=== Expected Behavior:
`.filter(...)` does not throw RecursionError and produce an SQL query
equivalent to
{{{#!sql
SELECT
"company"."id"
FROM
"company"
INNER JOIN "employee" ON (
"company"."id" = "employee"."company_id"
)
INNER JOIN "mission" specific_mission ON (
"employee"."id" = specific_mission."employee_id"
AND specific_mission."name" = 'specific'
)
WHERE
specific_mission."deadline" < '2025-10-23 11:07:06.224821+02:00'
}}}

=== Actual Behavior:
`.filter(...)` throws RecursionError

=== Analysis:
My understanding `Query.join()` fails to reuse the join for first reverse
relation, leading to the `FilteredRelation` being resolved again and
again.
Loop is:
- `FilteredRelation.resolve_expression()`
- `Query.build_filter()`
- `Query._add_q()`
- `Query.build_filter()`
- `Query.setup_joins()`
- `Query.join()`
- `FilteredRelation.resolve_expression()`

=== Workaround:
{{{#!python
qs = Company.objects.alias(
_employees=FilteredRelation("employees"),
specific_mission=FilteredRelation(
"_employees__missions",
condition=Q(_employees__missions__name="specific"),
),
)
qs = qs.filter(specific_mission__deadline__lt=localtime())
list(qs)
}}}


=== Related tickets:
#36109: Related to `FilteredRelation` and `RecursionError` (uses mentioned
workaround)
#34957: Probably the same underlying issue, ticket was closed due to lacks
reproduction details
--
Ticket URL: <https://code.djangoproject.com/ticket/36682>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------

Reporter: REGNIER Guillaume | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* stage: Unreviewed => Accepted

Comment:

Thank you for your reported, reproduced against
5e2bbebed9f36bb9d15f168444e7982287761877.
--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:1>

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------
Reporter: REGNIER Guillaume | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by REGNIER Guillaume:

Old description:

New description:

My understanding is that `Query.join()` fails to reuse the join for the

--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:2>

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------

Reporter: REGNIER Guillaume | Owner: Augusto
| Pontes
Type: Bug | Status: assigned

Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Augusto Pontes):

* has_patch: 0 => 1
* needs_docs: 0 => 1
* needs_tests: 0 => 1
* owner: (none) => Augusto Pontes
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:3>

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------
Reporter: REGNIER Guillaume | Owner: Augusto
| Pontes
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Augusto Pontes):

Hi, so i have a suggestion of how fixing it, i sent a PR, have a look:

https://github.com/django/django/pull/20010
--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:4>

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------
Reporter: REGNIER Guillaume | Owner: Augusto
| Pontes
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:

Keywords: FilteredRelation | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Augusto Pontes):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:5>

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------
Reporter: REGNIER Guillaume | Owner: Augusto
| Pontes
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:

Keywords: FilteredRelation | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* needs_better_patch: 0 => 1
* needs_docs: 1 => 0
* stage: Ready for checkin => Accepted

Comment:

You [https://docs.djangoproject.com/en/5.2/internals/contributing
/triaging-tickets/#ready-for-checkin can't mark your own patch RFC].

More over the proposed patch has several problems that should be addressed
before it can be considered for final review.
--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:6>

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------
Reporter: REGNIER Guillaume | Owner: Augusto
| Pontes
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Augusto Pontes):

Replying to [comment:6 Simon Charette]:

> You [https://docs.djangoproject.com/en/5.2/internals/contributing
/triaging-tickets/#ready-for-checkin can't mark your own patch RFC].
>
> More over the proposed patch has several problems that should be
addressed before it can be considered for final review.

Ok
--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:7>

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------
Reporter: REGNIER Guillaume | Owner: Augusto
| Pontes
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Augusto Pontes):

Replying to [comment:6 Simon Charette]:
> You [https://docs.djangoproject.com/en/5.2/internals/contributing
/triaging-tickets/#ready-for-checkin can't mark your own patch RFC].
>
> More over the proposed patch has several problems that should be
addressed before it can be considered for final review.

I accept suggestions on the PR of how can i improve to solve this problem
--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:8>

[Django]

#36682: RecursionError when traversing several reverse relations with
FilteredRelation
-------------------------------------+-------------------------------------
Reporter: REGNIER Guillaume | Owner: Augusto
| Pontes
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: FilteredRelation | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Augusto Pontes):

Replying to [comment:6 Simon Charette]:
> You [https://docs.djangoproject.com/en/5.2/internals/contributing
/triaging-tickets/#ready-for-checkin can't mark your own patch RFC].
>
> More over the proposed patch has several problems that should be
addressed before it can be considered for final review.

I saw your comments, i will read them
--
Ticket URL: <https://code.djangoproject.com/ticket/36682#comment:9>