[Django] #31990: Queryset "ordered" property is misleading if "annotate" function is used

50 views
Skip to first unread message

Django

unread,
Sep 9, 2020, 8:51:53 AM9/9/20
to django-...@googlegroups.com
#31990: Queryset "ordered" property is misleading if "annotate" function is used
-------------------------------------+-------------------------------------
Reporter: Julien | Owner: nobody
Dutriaux |
Type: Bug | Status: new
Component: Database | Version: 3.1
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Using the `annotate` function on a queryset doesn't keep the default
ordering set in model's meta class.

A property should say whether the queryset will be ordered or not. I
wanted to use the `qs.ordered` property for this but it seems to stay
truthy, even if the resulting SQL query will not have an ORDER BY clause.

Example:
{{{
qs = Foo.objects.all()

# SQL => 'SELECT "foo_foo"."uuid", "foo_foo"."name" FROM "foo_foo" ORDER
BY "foo_foo"."name" ASC'

qs.ordered # => True
qs.query.default_ordering # => True

############################################

qs2 = Foo.objects.annotate(Count("pk")).all()

# SQL => 'SELECT "foo_foo"."uuid", "foo_foo"."name",
COUNT("foo_foo"."uuid") AS "pk__count" FROM "foo_foo" GROUP BY
"foo_foo"."uuid"'

qs2.ordered # => True
qs2.query.default_ordering # => True
}}}

If it can help : I'm using PostgreSQL

--
Ticket URL: <https://code.djangoproject.com/ticket/31990>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Sep 10, 2020, 1:48:15 AM9/10/20
to django-...@googlegroups.com
#31990: QuerySet.ordered property is incorrect after annotate().
-------------------------------------+-------------------------------------
Reporter: Julien Dutriaux | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 3.1
(models, ORM) | Resolution:
Severity: Normal | worksforme
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by felixxm):

* status: new => closed
* resolution: => worksforme


Comment:

Thanks for this report, however `QuerySet.ordered` works for me, see
[https://github.com/django/django/blob/502e75f9ed5476ffe8229109acf0c23999d4b533/tests/queries/tests.py#L2082-L2085
tests].

--
Ticket URL: <https://code.djangoproject.com/ticket/31990#comment:1>

Django

unread,
Sep 10, 2020, 3:30:26 AM9/10/20
to django-...@googlegroups.com
#31990: QuerySet.ordered property is incorrect after annotate().
-------------------------------------+-------------------------------------
Reporter: Julien Dutriaux | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Julien Dutriaux):

* status: closed => new
* resolution: worksforme =>


Comment:

Replying to [comment:1 felixxm]:


> Thanks for this report, however `QuerySet.ordered` works for me, see
[https://github.com/django/django/blob/502e75f9ed5476ffe8229109acf0c23999d4b533/tests/queries/tests.py#L2082-L2085
tests].

Thanks for your answer but I've just checked again as it is in my example
and `ordered` still returns True if an annotation is applied.
As said earlier, my model has a default ordering in the Meta class, the
one in your unit test doesn't have.
If I modify the `Annotation` model to add a default ordering, the test
doesn't pass anymore.
It can be the intended behavior but it just sounds misleading to me
(having an `ordered` property sets to `True` while the results will
**not** be ordered).

--
Ticket URL: <https://code.djangoproject.com/ticket/31990#comment:2>

Django

unread,
Sep 10, 2020, 4:24:33 AM9/10/20
to django-...@googlegroups.com
#31990: QuerySet.ordered property is incorrect for GROUP BY queries on models with
Meta.ordering.

-------------------------------------+-------------------------------------
Reporter: Julien Dutriaux | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by felixxm):

* cc: Ramiro Morales (added)
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted


Comment:

Thanks for clarification, `Meta.ordering` doesn't affect `GROUP BY`
queries after 0ddb4ebf7bfcc4730c80a772dd146a49ef6895f6 (which was
deprecated in 1b1f64ee5a78cc217fead52cbae23114502cf564). We should adjust
`QuerySet.ordered`.

--
Ticket URL: <https://code.djangoproject.com/ticket/31990#comment:3>

Django

unread,
Sep 10, 2020, 4:29:56 AM9/10/20
to django-...@googlegroups.com
#31990: QuerySet.ordered property is incorrect for GROUP BY queries on models with
Meta.ordering.
-------------------------------------+-------------------------------------
Reporter: Julien Dutriaux | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by felixxm):

* Attachment "test-31990.diff" added.

Regression test.

Django

unread,
Sep 14, 2020, 3:29:44 AM9/14/20
to django-...@googlegroups.com
#31990: QuerySet.ordered property is incorrect for GROUP BY queries on models with
Meta.ordering.
-------------------------------------+-------------------------------------
Reporter: Julien Dutriaux | Owner: felixxm
Type: Bug | Status: assigned

Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by felixxm):

* owner: nobody => felixxm
* status: new => assigned


--
Ticket URL: <https://code.djangoproject.com/ticket/31990#comment:4>

Django

unread,
Sep 14, 2020, 3:49:36 AM9/14/20
to django-...@googlegroups.com
#31990: QuerySet.ordered property is incorrect for GROUP BY queries on models with
Meta.ordering.
-------------------------------------+-------------------------------------
Reporter: Julien Dutriaux | Owner: felixxm
Type: Bug | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by felixxm):

* has_patch: 0 => 1


Comment:

[https://github.com/django/django/pull/13417 PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/31990#comment:5>

Django

unread,
Sep 14, 2020, 2:08:31 PM9/14/20
to django-...@googlegroups.com
#31990: QuerySet.ordered property is incorrect for GROUP BY queries on models with
Meta.ordering.
-------------------------------------+-------------------------------------
Reporter: Julien Dutriaux | Owner: felixxm
Type: Bug | Status: closed

Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by GitHub <noreply@…>):

* status: assigned => closed
* resolution: => fixed


Comment:

In [changeset:"e11d05e0b488a3ff2b3c9d8f2e1e50f471750d6e" e11d05e0]:
{{{
#!CommitTicketReference repository=""
revision="e11d05e0b488a3ff2b3c9d8f2e1e50f471750d6e"
Fixed #31990 -- Fixed QuerySet.ordered for GROUP BY queries on models with
Meta.ordering.

Regression in 0ddb4ebf7bfcc4730c80a772dd146a49ef6895f6.

Thanks Julien Dutriaux for the report.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/31990#comment:6>

Django

unread,
Sep 14, 2020, 2:08:55 PM9/14/20
to django-...@googlegroups.com
#31990: QuerySet.ordered property is incorrect for GROUP BY queries on models with
Meta.ordering.
-------------------------------------+-------------------------------------
Reporter: Julien Dutriaux | Owner: felixxm
Type: Bug | Status: closed
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"a3bb80dc3186bd4d11b7864268beb88595ab00fd" a3bb80dc]:
{{{
#!CommitTicketReference repository=""
revision="a3bb80dc3186bd4d11b7864268beb88595ab00fd"
[3.1.x] Fixed #31990 -- Fixed QuerySet.ordered for GROUP BY queries on
models with Meta.ordering.

Regression in 0ddb4ebf7bfcc4730c80a772dd146a49ef6895f6.

Thanks Julien Dutriaux for the report.

Backport of e11d05e0b488a3ff2b3c9d8f2e1e50f471750d6e from master
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/31990#comment:7>

Reply all
Reply to author
Forward
0 new messages