[Django] #37312: QuerySet.in_bulk() drops annotations selected by values() and values

Django

unread,

8:42 AM (14 hours ago) 8:42 AM

to django-...@googlegroups.com

#37312: QuerySet.in_bulk() drops annotations selected by values() and
values_list().
-------------------------------------+-------------------------------------
Reporter: Yassin Bahri | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 6.1 | Severity: Normal
Keywords: in_bulk values | Triage Stage:
annotations | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
`QuerySet.in_bulk()` silently drops annotations selected by `.values()` or
`.values_list()` when the field used as the dictionary key isn't included
in the selected fields.

This is reproducible on current `main` at
`5babd2e21ac0877d66c5452da17b97608b337fba` and on `stable/6.1.x` at
`4b0185a5fb`, using SQLite.

Given these models:

{{{#!python
class Author(models.Model):
name = models.CharField(max_length=100)

class Article(models.Model):
headline = models.CharField(max_length=100)
author = models.ForeignKey(Author, models.CASCADE)
}}}

The following reproduces the issue:

{{{#!python
from django.db.models import F

author = Author.objects.create(name="Author 1")
article = Article.objects.create(
headline="Article 1",
author=author,
)

result = (
Article.objects.annotate(author_name=F("author__name"))
.values("headline", "author_name")
.in_bulk([article.pk])
)

print(result)
}}}

The actual result omits the selected annotation:

{{{#!python
{
article.pk: {
"headline": "Article 1",
}
}
}}}

The expected result is:

{{{#!python
{
article.pk: {
"headline": "Article 1",
"author_name": "Author 1",
}
}
}}}

The same problem occurs with `.values_list()`:

{{{#!python
result = (
Article.objects.annotate(author_name=F("author__name"))
.values_list("headline", "author_name")
.in_bulk([article.pk])
)
}}}

The actual value is:

{{{#!python
{article.pk: ("Article 1",)}
}}}

instead of:

{{{#!python
{article.pk: ("Article 1", "Author 1")}
}}}

Support for chaining `in_bulk()` after `values()` and `values_list()` was
added in #36605 by [https://github.com/django/django/commit/1820d35b17f
1820d35].

When `in_bulk()` internally adds the missing key field to the projection,
it rebuilds the projection from `query.values_select`. Selected
annotations are stored separately and are therefore omitted from the
rebuilt projection.
--
Ticket URL: <https://code.djangoproject.com/ticket/37312>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

11:16 AM (12 hours ago) 11:16 AM

to django-...@googlegroups.com

#37312: QuerySet.in_bulk() drops annotations selected by values() and
values_list().
-------------------------------------+-------------------------------------

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* cc: Adam Johnson (added)
* stage: Unreviewed => Accepted

--
Ticket URL: <https://code.djangoproject.com/ticket/37312#comment:1>

Django

unread,

11:17 AM (12 hours ago) 11:17 AM

to django-...@googlegroups.com

#37312: QuerySet.in_bulk() drops annotations selected by values() and
values_list().
-------------------------------------+-------------------------------------
Reporter: Yassin Bahri | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 6.1
(models, ORM) |

Severity: Release blocker | Resolution:

Keywords: in_bulk values | Triage Stage: Accepted
annotations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Jacob Walls):

* severity: Normal => Release blocker

--
Ticket URL: <https://code.djangoproject.com/ticket/37312#comment:2>

Django

unread,

11:19 AM (12 hours ago) 11:19 AM

to django-...@googlegroups.com

#37312: QuerySet.in_bulk() drops annotations selected by values() and
values_list().
-------------------------------------+-------------------------------------

Reporter: Yassin Bahri | Owner: Yassin
| Bahri
Type: Bug | Status: assigned

Changes (by Yassin Bahri):

* owner: (none) => Yassin Bahri
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/37312#comment:3>

Django

unread,

11:34 AM (12 hours ago) 11:34 AM

to django-...@googlegroups.com

#37312: QuerySet.in_bulk() drops annotations selected by values() and
values_list().
-------------------------------------+-------------------------------------
Reporter: Yassin Bahri | Owner: Yassin
| Bahri
Type: Bug | Status: assigned
Component: Database layer | Version: 6.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: in_bulk values | Triage Stage: Accepted
annotations |

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Yassin Bahri):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/37312#comment:4>

Django

unread,

1:58 PM (9 hours ago) 1:58 PM

to django-...@googlegroups.com

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/37312#comment:5>

Reply all

Reply to author

Forward

[Django] #37312: QuerySet.in_bulk() drops annotations selected by values() and values_list().

Django

Django

Django

Django

Django

Django