[Django] #35064: Window.order_by decimal field is broken on SQLite

30 views
Skip to first unread message

Django

unread,
Dec 27, 2023, 1:50:01 PM12/27/23
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon | Owner: nobody
Charette |
Type: Bug | Status: new
Component: Database | Version: 3.2
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Initially reported
[https://discord.com/channels/856567261900832808/1180704302847766538/1180704302847766538
on the Discord] and demonstrated in
[https://github.com/Quoates/django_rank_order_by this Django project].

{{{#!python
class RankTest(models.Model):
name = models.CharField(max_length=30)
category = models.CharField(max_length=30)
rating = models.DecimalField(max_digits=8, decimal_places=5)

list(
RankTest.objects.annotate(
rank=Window(
expression=Rank(),
order_by='rating'
)
)
)
}}}

The solution implemented in #31723
(71d10ca8c90ccc1fd0ccd6683716dd3c3116ae6a) wish addressed the improper of
casting for `Window.expression` caused some problematic one for `order_by`
and likely `partition_by` as well.

--
Ticket URL: <https://code.djangoproject.com/ticket/35064>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Dec 27, 2023, 2:33:14 PM12/27/23
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* stage: Unreviewed => Accepted


--
Ticket URL: <https://code.djangoproject.com/ticket/35064#comment:1>

Django

unread,
Dec 29, 2023, 1:04:17 AM12/29/23
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: assigned

Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

* owner: nobody => Simon Charette
* status: new => assigned


--
Ticket URL: <https://code.djangoproject.com/ticket/35064#comment:2>

Django

unread,
Dec 29, 2023, 1:11:29 AM12/29/23
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

* has_patch: 0 => 1


--
Ticket URL: <https://code.djangoproject.com/ticket/35064#comment:3>

Django

unread,
Dec 29, 2023, 2:17:33 AM12/29/23
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* stage: Accepted => Ready for checkin


--
Ticket URL: <https://code.djangoproject.com/ticket/35064#comment:4>

Django

unread,
Dec 29, 2023, 3:58:57 AM12/29/23
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"90d365d869924d503565eb1ccdc24077d60faf0c" 90d365d8]:
{{{
#!CommitTicketReference repository=""
revision="90d365d869924d503565eb1ccdc24077d60faf0c"
Refs #35064 -- Made OrderableAggMixin avoid creating empty OrderByList.

This paves the way for making OrderByList a simple shim over
ExpressionList which requires at least a single item to be provided.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/35064#comment:5>

Django

unread,
Dec 29, 2023, 3:58:57 AM12/29/23
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: closed

Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):

* status: assigned => closed
* resolution: => fixed


Comment:

In [changeset:"e16d0c176e9b89628cdec5e58c418378c4a2436a" e16d0c1]:
{{{
#!CommitTicketReference repository=""
revision="e16d0c176e9b89628cdec5e58c418378c4a2436a"
Fixed #35064 -- Fixed Window(order_by) crash with DecimalFields on SQLite.

This avoids cast of Window(order_by) for DecimalFields on SQLite.

This was achieved by piggy-backing ExpressionList which already
implements a specialized as_sqlite() method to override the inherited
behaviour of Func through SQLiteNumericMixin.

Refs #31723.

Thanks Quoates for the report.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/35064#comment:6>

Django

unread,
Aug 13, 2024, 5:26:27 AM8/13/24
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"602fe961e6834d665f2359087a1272e9f9806b71" 602fe96]:
{{{#!CommitTicketReference repository=""
revision="602fe961e6834d665f2359087a1272e9f9806b71"
Fixed #35665 -- Fixed a crash when passing an empty order_by to Window.

This also caused un-ordered sliced prefetches to crash as they rely on
Window.

Regression in e16d0c176e9b89628cdec5e58c418378c4a2436a that made
OrderByList
piggy-back ExpressionList without porting the empty handling that the
latter
provided.

Supporting explicit empty ordering on Window functions and slicing is
arguably
a foot-gun design due to how backends will return undeterministic results
but
this is a problem that requires a larger discussion.

Refs #35064.

Thanks Andrew Backer for the report and Mariusz for the review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35064#comment:7>

Django

unread,
Aug 13, 2024, 5:28:38 AM8/13/24
to django-...@googlegroups.com
#35064: Window.order_by decimal field is broken on SQLite
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"df236b0bcbbf1f54dfe6acc7761cd81b76ebf2cc" df236b0]:
{{{#!CommitTicketReference repository=""
revision="df236b0bcbbf1f54dfe6acc7761cd81b76ebf2cc"
[5.1.x] Fixed #35665 -- Fixed a crash when passing an empty order_by to
Window.

This also caused un-ordered sliced prefetches to crash as they rely on
Window.

Regression in e16d0c176e9b89628cdec5e58c418378c4a2436a that made
OrderByList
piggy-back ExpressionList without porting the empty handling that the
latter
provided.

Supporting explicit empty ordering on Window functions and slicing is
arguably
a foot-gun design due to how backends will return undeterministic results
but
this is a problem that requires a larger discussion.

Refs #35064.

Thanks Andrew Backer for the report and Mariusz for the review.

Backport of 602fe961e6834d665f2359087a1272e9f9806b71 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35064#comment:8>
Reply all
Reply to author
Forward
0 new messages