[Django] #35042: Queryset count does not work after union

[Django]

#35042: Queryset count does not work after union
-----------------------------------------+------------------------
Reporter: Marcin | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 4.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
While upgrading to Django 4 I've found a bug with `count()` which can be
replicated using the following code. Notice that evaluating the queryset
works fine, the issue is that for some reason, when counting django is not
including `total_count` column from the second queryset in the query and
this is causing union to fail. In Django 3 the code works fine and
produces a correct SQL. I've attached an example project which can be used
to reproduce the bug, with command `tox -e ALL`.

models.py
{{{#!python
class Foo(models.Model):
name = models.TextField()

class Bar(models.Model):
foo = models.ForeignKey(Foo, on_delete=models.CASCADE)
count = models.IntegerField()
}}}

tests.py
{{{#!python
qs1 = Foo.objects.all().annotate(
_count=models.Sum('bar__count')
).values('name').annotate(
total_count=models.F('_count')
).values('name', 'total_count')

qs2 = Bar.objects.all().annotate(
name=models.F('foo__name'),
total_count=models.Value(0, output_field=models.IntegerField())
).values('name', 'total_count')

self.assertEqual(list(qs1.union(qs2, all=True)), [])
self.assertEqual(qs1.union(qs2, all=True).count(), 0)
}}}

Exception thrown in Django 4+

{{{
Traceback (most recent call last):
File "/home/user/foo/bar/tests.py", line 21, in test_recreate_bug
self.assertEqual(qs1.union(qs2, all=True).count(), 0)
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/models/query.py", line 618, in count
return self.query.get_count(using=self.db)
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/models/sql/query.py", line 616, in get_count
return obj.get_aggregation(using, {"__count": Count("*")})["__count"]
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/models/sql/query.py", line 602, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
cursor.execute(sql, params)
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
File "/home/user/foo/.tox/django5/lib/python3.10/site-
packages/django/db/backends/sqlite3/base.py", line 328, in execute
return super().execute(query, params)
django.db.utils.OperationalError: SELECTs to the left and right of UNION
ALL do not have the same number of result columns
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/35042>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35042: Queryset count does not work after union

-------------------------------+--------------------------------------

Reporter: Marcin | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 4.2

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------+--------------------------------------
Changes (by Marcin):

* Attachment "foo.tar.gz" added.

minimal example