[Django] #33159: Missing table alias in generated query after used in subquery
Django
-------------------------------------+-------------------------------------
Reporter: Michal | Owner: nobody
Čihař |
Type: Bug | Status: new
Component: Database | Version: 4.0
layer (models, ORM) |
Severity: Release | Keywords:
blocker |
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
When running our tests against 4.0a1, it fails with:
{{{
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/site-
packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: missing FROM-clause entry for table "U0"
LINE 1: ...UNT(*) AS "__count" FROM "trans_component" WHERE ("U0"."repo...
}}}
It seems that the table alias is missing from the query. Full log can be
seen at
https://github.com/WeblateOrg/weblate/pull/6608/checks?check_run_id=3751233865
Reproducing outside the test case:
{{{
>>> from weblate.trans.models import Component
>>> from django.db.models import Q
>>> c = Component.objects.filter(Q(repo__in=('x', 'y')) |
Q(repo__endswith='xxxx'))
>>> c.count()
0
>>> Component.objects.filter(linked_component__in=c).count()
0
>>> c.count()
Traceback (most recent call last):
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: missing FROM-clause entry for table "U0"
LINE 1: ...UNT(*) AS "__count" FROM "trans_component" WHERE ("U0"."repo...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/models/query.py", line 416, in count
return self.query.get_count(using=self.db)
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/models/sql/query.py", line 515, in get_count
number = obj.get_aggregation(using, ['__count'])['__count']
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/models/sql/query.py", line 500, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/models/sql/compiler.py", line 1198, in execute_sql
cursor.execute(sql, params)
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/backends/utils.py", line 98, in execute
return super().execute(sql, params)
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False,
executor=self._execute)
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/nijel/weblate/weblate/.venv/lib/python3.9/site-
packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: missing FROM-clause entry for table "U0"
LINE 1: ...UNT(*) AS "__count" FROM "trans_component" WHERE ("U0"."repo...
^
}}}
Note:
* The queryset has to consist of multiple Q
* Calling count on the queryset works until it is used in a subquery
--
Ticket URL: <https://code.djangoproject.com/ticket/33159>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Keryn Knight (added)
* stage: Unreviewed => Accepted
Comment:
Thanks for the report!
Regression in e441847ecae99dd1ccd0d9ce76dbcff51afa863c, TBH I was pretty
sure that this would introduce some regression, now we will have a proper
regression test.
--
Ticket URL: <https://code.djangoproject.com/ticket/33159#comment:1>
Django
-------------------------------------+-------------------------------------
| Felisiak
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Mariusz Felisiak
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/33159#comment:2>
Django
-------------------------------------+-------------------------------------
| Felisiak
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/14916 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/33159#comment:3>
Django
-------------------------------------+-------------------------------------
| Felisiak
Component: Database layer | Version: 4.0
(models, ORM) |
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"903aaa35e5ceaa33bfc9b19b7f6da65ce5a91dd4" 903aaa3]:
{{{
#!CommitTicketReference repository=""
revision="903aaa35e5ceaa33bfc9b19b7f6da65ce5a91dd4"
Fixed #33159 -- Reverted "Fixed #32970 -- Changed WhereNode.clone() to
create a shallow copy of children."
This reverts commit e441847ecae99dd1ccd0d9ce76dbcff51afa863c.
A shallow copy is not enough because querysets can be reused and
evaluated in nested nodes, which shouldn't mutate JOIN aliases.
Thanks Michal Čihař for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33159#comment:4>
Django
-------------------------------------+-------------------------------------
| Felisiak
Type: Bug | Status: closed
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"93a42d43a6995993b9bbcb743ab3c2a2b8414ebd" 93a42d43]:
{{{
#!CommitTicketReference repository=""
revision="93a42d43a6995993b9bbcb743ab3c2a2b8414ebd"
[4.0.x] Fixed #33159 -- Reverted "Fixed #32970 -- Changed
WhereNode.clone() to create a shallow copy of children."
This reverts commit e441847ecae99dd1ccd0d9ce76dbcff51afa863c.
A shallow copy is not enough because querysets can be reused and
evaluated in nested nodes, which shouldn't mutate JOIN aliases.
Thanks Michal Čihař for the report.
Backport of 903aaa35e5ceaa33bfc9b19b7f6da65ce5a91dd4 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33159#comment:5>