[Django] #29254: Improve paginator count performance by removing select

Django

unread,

Mar 24, 2018, 7:46:14 AM3/24/18

to django-...@googlegroups.com

#29254: Improve paginator count performance by removing select_related from the
query
-------------------------------------+-------------------------------------
Reporter: hakib | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core | Version: 2.0
(Other) |
Severity: Normal | Keywords: pagination
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-------------------------------------+-------------------------------------
The default paginator is initiated with an `object_list` and uses `.count`
or ` len` on it to get the total number of rows required to calculate the
current page, next page etc...

Most of the time the object_list will be a QuerySet. When this is the case
we can get an easy performace boost by removing any select_related from
the query. select_related are adding an outer join so it cannot effect the
total number of rows in query set - **select_related is basically dead
weight if you only want to count the rows**.

`Paginator.count` is pretty simple:

{{{
def count(self):
"""
Returns the total number of objects, across all pages.
"""
try:
return self.object_list.count()
except (AttributeError, TypeError):
# AttributeError if object_list has no count() method.
# TypeError if object_list.count() requires arguments
# (i.e. is of type list).
return len(self.object_list)
}}}

By changing to this:

{{{
return self.object_list.select_related(None).count()
}}}

**we can significantly improve the performace of the count when the
original query has many select_related models** (as often seen in admin
pages with admin_select_related and views that show tables of data).

Only reservation I have is that `Paginator` is currently designed to
support any kind of object that implements either `*.count` or `len(*)` -
the tests also include some tests for random objects that implement count.
This can easily be solved using `isinstance(object_list, QuerySet)`,
sniffing for `hasatrr(select_related, object_list)` or similar to how it's
implemented today using `AttributeError`.

--
Ticket URL: <https://code.djangoproject.com/ticket/29254>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Mar 24, 2018, 10:35:07 AM3/24/18

to django-...@googlegroups.com

#29254: Improve paginator count performance by removing select_related from the
query
-------------------------------------+-------------------------------------
Reporter: hakib | Owner: nobody

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by hakib):

* status: new => closed
* resolution: => invalid

--
Ticket URL: <https://code.djangoproject.com/ticket/29254#comment:1>

Django

unread,

Jul 17, 2018, 12:41:20 PM7/17/18

to django-...@googlegroups.com

#29254: Improve paginator count performance by removing select_related from the
query
-------------------------------------+-------------------------------------
Reporter: hakib | Owner: nobody

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

[Django] #29254: Improve paginator count performance by removing select_related from the query

Django

Django

Django