[Django] #28897: FieldError when performing custom action in Admin Interface IF data is sorted by an annotated field
Django
sorted by an annotated field
-------------------------------------+-------------------------------------
Reporter: coltonbh | Owner: nobody
Type: Bug | Status: new
Component: | Version: 2.0
contrib.admin | Keywords: Admin Interface,
Severity: Normal | Custom Action, Annotated Field,
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
A FieldError results if I try to perform a custom a custom action on data
in the admin interface IF that data is sorted by an annotated field. The
action is obviously not acting on an annotated field as those fields do
not exist in the database, but raises a FieldError saying it 'Cannot
resolve keyword 'field_that_data_is_currently_sorted_by' into field.' and
then lists the choices for fields (all of which are the raw model fields,
not the annotated fields in the custom queryset).
My admin model:
{{{
@admin.register(Position)
class PositionAdmin(admin.ModelAdmin):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def get_queryset(self, request):
qs = super(PositionAdmin,
self).get_queryset(request).prefetch_related(
'orders').select_related(
'portfolio', 'current_position').select_related(
'current_position').annotate(
Count('orders', distinct=True),
Count('short_requests'),
total_position_value=ExpressionWrapper(
F('price') * F('quantity'),
output_field=IntegerField()),
diff_shares=Coalesce(
F('quantity')
- F('current_position__quantity'),
F('quantity')),
diff_value=ExpressionWrapper(
F('diff_shares') * F('price'),
output_field=IntegerField()),
orders_with_errors=Count(
Case(When(~Q(orders__error=''), then=1))),
non_accepted_orders=Count(
Case(When(Q(orders__status=''), then=1))),
order_today=Count(
Case(When(
orders__created_at__gte=_brokerage_today_start(),
then=1))))
return qs
# Custom Action
def approve_position_for_trading(self, request, queryset):
try:
num_pos_approved = queryset.update(approved=True)
except FieldError:
self.message_user(
request, "You cannot perform actions when you have sorted
by "
"this column. Please remove your column sortings and then
try "
"your action again.", level=messages.ERROR)
else:
if num_pos_approved == 1:
message_bit = "1 position was was"
else:
message_bit = "%s positions were" % num_pos_approved
self.message_user(
request, "%s successfully approved for trading." %
message_bit)
}}}
I had to write code to handle the error so that the user will know how to
proceed. However, this seems like bad behavior. Django can successfully
perform that action (changing the 'Approved' field to 'True') on the data
if it is sorted by any core model field, or if sorted by no field at all.
However, if the data is sorted by a field resulting from the annotation of
my queryset, then I get the FieldError when I try to perform the action.
This seems like a bug.
--
Ticket URL: <https://code.djangoproject.com/ticket/28897>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
sorted by an annotated field
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 2.0
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage:
Custom Action, Annotated Field, | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sergey Fedoseev):
Could you provide traceback and try minimize your example?
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:1>
Django
sorted by an annotated field
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 2.0
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage:
Custom Action, Annotated Field, | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Sergey Fedoseev (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:2>
Django
sorted by an annotated field
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: contrib.admin | Version: 2.0
Severity: Normal | Resolution: needsinfo
Keywords: Admin Interface, | Triage Stage:
Custom Action, Annotated Field, | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => needsinfo
Comment:
Yes, a more minimal (but complete, with models) example seems needed.
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:3>
Django
sorted by an annotated field
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 4.0
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage:
Custom Action, Annotated Field, | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* version: 2.0 => 4.0
* resolution: needsinfo =>
Comment:
Hey folks, I've run into this bug both on django 3.x and 4.x. Here is a
small showcase project:
https://github.com/salomvary/django_admin_action_bug
Steps to reproduce:
- Run the project
- Sign in to admin
- Create a few "Things"
- Go to Home › My_App › Things and select a Thing using the checkboxes
- Go to Actions > "Update Something"
- Click "Go"
You will see the following error page:
{{{
FieldError at /admin/my_app/thing/
Cannot resolve keyword 'other_things_count' into field. Choices are: id,
is_something, other_things
}}}
Few things to note:
- Removing queryset.order_by eliminates the problem
- Removing annotations that involve joins also eliminates the problem
Stack trace:
{{{
Internal Server Error: /admin/my_app/thing/
Traceback (most recent call last):
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/contrib/admin/options.py", line 622, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/views/decorators/cache.py", line 56, in
_wrapped_view_func
response = view_func(request, *args, **kwargs)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/contrib/admin/sites.py", line 236, in inner
return view(request, *args, **kwargs)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/contrib/admin/options.py", line 1736, in
changelist_view
response = self.response_action(request,
queryset=cl.get_queryset(request))
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/contrib/admin/options.py", line 1417, in
response_action
response = func(self, request, queryset)
File "/Users/mrc/Projects/django_admin_action_bug/my_app/admin.py", line
9, in update_something
queryset.update(is_something=True)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/query.py", line 790, in update
rows = query.get_compiler(self.db).execute_sql(CURSOR)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 1591, in
execute_sql
cursor = super().execute_sql(result_type)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 1189, in
execute_sql
sql, params = self.as_sql()
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 1526, in as_sql
self.pre_sql_setup()
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 1626, in
pre_sql_setup
super().pre_sql_setup()
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 60, in
pre_sql_setup
order_by = self.get_order_by()
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 389, in
get_order_by
for expr, is_ref in self._order_by_pairs():
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 373, in
_order_by_pairs
yield from self.find_ordering_name(
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 767, in
find_ordering_name
field, targets, alias, joins, path, opts, transform_function =
self._setup_joins(pieces, opts, alias)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/compiler.py", line 807, in
_setup_joins
field, targets, opts, joins, path, transform_function =
self.query.setup_joins(pieces, opts, alias)
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/query.py", line 1604, in setup_joins
path, final_field, targets, rest = self.names_to_path(
File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9
/site-packages/django/db/models/sql/query.py", line 1522, in names_to_path
raise FieldError("Cannot resolve keyword '%s' into field. "
django.core.exceptions.FieldError: Cannot resolve keyword
'other_things_count' into field. Choices are: id, is_something,
other_things
[09/Dec/2021 17:00:45] "POST /admin/my_app/thing/ HTTP/1.1" 500 154267
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:4>
Django
sorted by an annotated field
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 4.0
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage:
Custom Action, Annotated Field, | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Sergey Fedoseev (removed)
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:5>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* component: contrib.admin => Database layer (models, ORM)
* stage: Unreviewed => Accepted
Comment:
Thanks for details! `QuerySet.update()` removes all annotations (see
a84344bc539c66589c8d4fe30c6ceaecf8ba1af3) that's why `order_by()` raises a
`FieldError`. We should probably clear ordering in `QuerySet.update()` or
at least remove all annotated fields from it, e.g.
{{{
diff --git a/django/db/models/query.py b/django/db/models/query.py
index fb6639793a..1db9de0b5c 100644
--- a/django/db/models/query.py
+++ b/django/db/models/query.py
@@ -790,6 +790,7 @@ class QuerySet:
query = self.query.chain(sql.UpdateQuery)
query.add_update_values(kwargs)
# Clear any annotations so that they won't be present in
subqueries.
+ query.clear_ordering()
query.annotations = {}
with transaction.mark_for_rollback_on_error(using=self.db):
rows = query.get_compiler(self.db).execute_sql(CURSOR)
}}}
or
{{{
diff --git a/django/db/models/query.py b/django/db/models/query.py
index fb6639793a..90a0041d66 100644
--- a/django/db/models/query.py
+++ b/django/db/models/query.py
@@ -790,6 +790,9 @@ class QuerySet:
query = self.query.chain(sql.UpdateQuery)
query.add_update_values(kwargs)
# Clear any annotations so that they won't be present in
subqueries.
+ query.order_by = tuple(
+ col for col in query.order_by if col not in query.annotations
+ )
query.annotations = {}
with transaction.mark_for_rollback_on_error(using=self.db):
rows = query.get_compiler(self.db).execute_sql(CURSOR)
}}}
As a workaround you can clear ordering in your actions, e.g.
{{{#!python
def update_something(admin, request, queryset):
queryset.order_by().update(is_something=True)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:6>
Django
field.
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Márton Salomváry):
Thanks for the workaround, I can confirm it makes the exception go away.
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:7>
Django
field.
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Saidblanchette):
Hi, guys I'd love to work in this, any suggestions ! I need help to
understand this more and what can you do next
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:8>
Django
field.
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Saidblanchette (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:9>
Django
field.
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
Saeed, Have you seen my
[https://code.djangoproject.com/ticket/28897#comment:6 comment]?
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:10>
Django
field.
-------------------------------------+-------------------------------------
| Wobrock
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: David Wobrock (added)
* owner: nobody => David Wobrock
* has_patch: 0 => 1
* status: new => assigned
Comment:
The suggested code seems to fix it indeed :)
I could only reproduce when annotating/ordering on a ManyToManyField.
I used the snippet that keeps some `order_by` and doesn't clear it
entirely ¯\_(ツ)_/¯
[https://github.com/django/django/pull/15747 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:11>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:12>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:13>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:14>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Accepted
Custom Action, Annotated Field, |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:15>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Custom Action, Annotated Field, | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:16>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Admin Interface, | Triage Stage: Ready for
Custom Action, Annotated Field, | checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"f4680a112d01d85540411673eade31f37712d0a6" f4680a11]:
{{{
#!CommitTicketReference repository=""
revision="f4680a112d01d85540411673eade31f37712d0a6"
Refs #28897 -- Added test for QuerySet.update() on querysets ordered by
inline m2m annotation.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:17>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Component: Database layer | Version: 4.0
(models, ORM) |
Keywords: Admin Interface, | Triage Stage: Ready for
Custom Action, Annotated Field, | checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"3ef37a5245015f69a9b9f884ebc289a35d02c5f6" 3ef37a5]:
{{{
#!CommitTicketReference repository=""
revision="3ef37a5245015f69a9b9f884ebc289a35d02c5f6"
Fixed #28897 -- Fixed QuerySet.update() on querysets ordered by
annotations.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:18>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Type: Bug | Status: closed
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: Admin Interface, | Triage Stage: Ready for
Custom Action, Annotated Field, | checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"d44dc31fcb98935a9e1ec7c5fe60862d2ae4febc" d44dc31f]:
{{{
#!CommitTicketReference repository=""
revision="d44dc31fcb98935a9e1ec7c5fe60862d2ae4febc"
[4.1.x] Fixed #28897 -- Fixed QuerySet.update() on querysets ordered by
annotations.
Backport of 3ef37a5245015f69a9b9f884ebc289a35d02c5f6 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:20>
Django
field.
-------------------------------------+-------------------------------------
Reporter: Colton Hicks | Owner: David
| Wobrock
Type: Bug | Status: closed
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: Admin Interface, | Triage Stage: Ready for
Custom Action, Annotated Field, | checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"ccb243847e300283aca23af5eb3b6b4f2bdcfb28" ccb24384]:
{{{
#!CommitTicketReference repository=""
revision="ccb243847e300283aca23af5eb3b6b4f2bdcfb28"
[4.1.x] Refs #28897 -- Added test for QuerySet.update() on querysets
ordered by inline m2m annotation.
Backport of f4680a112d01d85540411673eade31f37712d0a6 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28897#comment:19>