[Django] #22503: Inconsistent behavior when a QuerySet is sliced
Django
----------------------------------------------+----------------------------
Reporter: ajaest | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: 1.6
Severity: Normal | Keywords: QuerySet,
Triage Stage: Unreviewed | filter, slice
Easy pickings: 0 | Has patch: 0
| UI/UX: 0
----------------------------------------------+----------------------------
I think that the current behavior of the QuerySet.slice method is not
consistent. Just an example:
{{{
def limited_filtering(max_results=10, *args, **kwargs){
"""
Returns a Queryset
"""
returns SomeModel.filter(*args, **kwargs)[:max_results]
}
# ...
def other_method_somewhere(queryset){
"""
Expects a Queryset of SomeModel and returns the same Queryset filtered
again
"""
return queryset.filter(some_other_criteria)
}
other_method_somewhere(limited_filtering(*args, **kwargs)); # raises
"AssertionError: Cannot filter a query once a slice has been taken."
}}}
Given the previous code, it is legit and fair that
`other_method_somewhere` expects, as in most situations, a filtrable
`QuerySet`. Not allowing a sliced `QuerySet` to be filtered after being
sliced adds an edge case difficult to handle when methods using a
`QuerySet` type are somehow semantically far away from each other (and
thus, they don't know if the other is returning an sliced QuerySet).
Without knowing the exact reasons why this was implemented this way, the
possible solutions I can come with are:
* The most natural one, to make the sliced `QuerySet`s filtrable.
* To make all the returned query sets in which `can_filter == False` to
return a new type of `QuerySet`, something like `InmutableQuerySet` in
order to take advantage of other methods related to them (`values_list`,
etc.).
* To make `QuerySet`'s slices return list or other type of collection (I
guess this is impossible without breaking backwards compatibility).
Actually, the last case is supposedly the way one would expect it to work
according to the documentation:
https://docs.djangoproject.com/en/dev/ref/models/querysets/#when-
querysets-are-evaluated
> '''Slicing'''. As explained in Limiting QuerySets, a QuerySet can be
sliced, using Python’s array-slicing syntax. Slicing an unevaluated
QuerySet usually returns another unevaluated QuerySet, but Django will
execute the database query if you use the “step” parameter of slice
syntax, and will return a list. Slicing a QuerySet that has been evaluated
(partially or fully) also returns a list.
By the way, there is also no reference in the documentation(or at least I
could not find it) to the public method `QuerySet.can_filter`.
--
Ticket URL: <https://code.djangoproject.com/ticket/22503>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: ajaest | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) | Resolution:
Severity: Normal | Triage Stage:
Keywords: QuerySet, filter, | Unreviewed
slice | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by timo):
* needs_better_patch: => 0
* type: Bug => New feature
* needs_tests: => 0
* needs_docs: => 0
Comment:
There was a recent [https://groups.google.com/d/topic/django-
developers/oqgtvFE5y84/discussion django-dev thread] about this. I have
not taken time to fully read it and determine if any consensus was
reached.
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:1>
Django
Reporter: ajaest | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) | Resolution:
Keywords: QuerySet, filter, | Needs documentation: 0
slice | Patch needs improvement: 0
Has patch: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by timo):
* stage: Unreviewed => Accepted
Comment:
If we don't do this, let's document why not.
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:2>
Django
Reporter: ajaest | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: QuerySet, filter, | Needs documentation: 0
Has patch: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by jorgecarleitao):
I will map the discussion in the thread to here: I believe the consensus
was that this cannot be done:
Assume `other_method_somewhere` uses a filter. The sequence would be
{{{
queryset.filter(first_clause)[:10].filter(second_clause)
}}}
Most people would assume (as pointed out in the thread) that the second
clause would be applied to the last 10 entries, when it is not. Doing that
is not possible in SQL without
subqueries and thus we explicitly forbid it.
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:3>
Django
Reporter: ajaest | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: QuerySet, filter, | Needs documentation: 0
Has patch: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by tchaumeny):
See also https://groups.google.com/forum/#!topic/django-
developers/ELqU2xt_Qo0 (sliced querysets not usable in a
`ModelChoiceField` because not filterable).
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:4>
Django
Reporter: ajaest | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: QuerySet, filter, | Needs documentation: 0
Has patch: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"327df551e89a505c5756becee97c40198f38aff2"]:
{{{
#!CommitTicketReference repository=""
revision="327df551e89a505c5756becee97c40198f38aff2"
Fixed #23817 -- Updated docs on QuerySet evaluation
Removed inaccurate info about partial evaluation after refs #18702.
Added information on modifying sliced QuerySets; refs #22503.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:5>
Django
Reporter: ajaest | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: QuerySet, filter, | Needs documentation: 0
Has patch: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"593353d8af5b4b8d1a1e627712fe68ed593961d0"]:
{{{
#!CommitTicketReference repository=""
revision="593353d8af5b4b8d1a1e627712fe68ed593961d0"
[1.7.x] Fixed #23817 -- Updated docs on QuerySet evaluation
Removed inaccurate info about partial evaluation after refs #18702.
Added information on modifying sliced QuerySets; refs #22503.
Backport of 327df551e89a505c5756becee97c40198f38aff2 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:7>
Django
Reporter: ajaest | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: QuerySet, filter, | Needs documentation: 0
Has patch: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"614dd44d0d2e5f8f12256835c3453f420f54c3b4"]:
{{{
#!CommitTicketReference repository=""
revision="614dd44d0d2e5f8f12256835c3453f420f54c3b4"
[1.6.x] Fixed #23817 -- Updated docs on QuerySet evaluation
Removed inaccurate info about partial evaluation after refs #18702.
Added information on modifying sliced QuerySets; refs #22503.
Backport of 327df551e89a505c5756becee97c40198f38aff2 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:6>
Django
Reporter: Luis A. Arce | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) |
Severity: Normal | Resolution:
Keywords: QuerySet, filter, | Triage Stage: Accepted
slice |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Дилян Палаузов):
Is this fixed? The last comments suggest so, but the status is "new".
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:8>
Django
Reporter: Luis A. Arce | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.6
(models, ORM) |
Severity: Normal | Resolution:
Keywords: QuerySet, filter, | Triage Stage: Accepted
slice |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Дилян Палаузов (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:9>
Django
Reporter: Luis A. Arce | Owner: nobody
(models, ORM) |
slice |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* resolution: => fixed
* status: new => closed
Comment:
I think the documentation edit fixed this.
--
Ticket URL: <https://code.djangoproject.com/ticket/22503#comment:10>