[Django] #26172: Don't query the database when filtering an already empty queryset
Django
----------------------------------------------+---------------------------
Reporter: seddonym | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Database layer (models, ORM) | Version: 1.9
Severity: Normal | Keywords: orm, queryset
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+---------------------------
If you have a queryset that has already been evaluated as empty, and you
filter (or exclude) it, Django will hit the database again, even though
the result will always be an empty queryset.
{{{
>>> queryset = MyModel.objects.filter(my_field='foo')
>>> bool(queryset)
# Hits the database - running bool will cache it
False
>>> queryset
[]
>>> queryset.filter(another_field='bar')
# Hits the database
[]
>>> queryset.exclude(another_field='bar')
# Hits the database
[]
}}}
The ORM should check to see if the queryset is empty before running any
further filters / excludes pointlessly.
--
Ticket URL: <https://code.djangoproject.com/ticket/26172>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: seddonym | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: orm, queryset | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by akaariai):
* status: new => closed
* needs_better_patch: => 0
* resolution: => wontfix
* needs_tests: => 0
* needs_docs: => 0
Comment:
I don't see this as possible. The reason is that the ORM can't know if the
queryset will be empty when the filter() clause is applied. Consider:
{{{
Foo.objects.all().delete()
qs = Foo.objects.filter(pk__gte=0)
bool(qs)
Foo.objects.create(pk=1)
qs = qs.filter(pk=1) # There is a match even if
Foo.objects.filter(pk__gte=0) didn't match anything!
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26172#comment:1>
Django
Reporter: seddonym | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Normal | Resolution: wontfix
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by seddonym):
Hmm, I see what you mean, but couldn't you make the same argument for
''any'' queryset caching? Consider:
{{{
>>> Foo.objects.all().delete()
>>> qs = Foo.objects.all()
>>> bool(qs)
False
>>> Foo.objects.create(pk=1)
>>> qs
# The cached results will be fetched
[]
}}}
I guess it depends on what you want the caching strategy to be. On the
one hand, returning an empty cached queryset is simply using the cache
that's available, which might be the right thing to do in some
circumstances. On the other, perhaps whenever filter() or exclude() are
applied, it should always invalidate the cache.
--
Ticket URL: <https://code.djangoproject.com/ticket/26172#comment:2>
Django
Reporter: seddonym | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Normal | Resolution: wontfix
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by akaariai):
The main reason why we use the caching approach we currently use is that
we have done it in this way always.
If we started from scratch, it might be we would opt for an explicit
approach - queries are only executed when qs.execute() is called. That way
the user would know exactly when the cache is created and cleared. But we
are extremely unlikely to change any of this now...
--
Ticket URL: <https://code.djangoproject.com/ticket/26172#comment:3>
Django
Reporter: seddonym | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Normal | Resolution: wontfix
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by seddonym):
Fair enough. After all, I guess there is nothing to stop someone making a
custom queryset with this behaviour.
Thanks for considering the ticket anyway, it's useful to know the
reasoning behind it.
--
Ticket URL: <https://code.djangoproject.com/ticket/26172#comment:4>