After:
{{{
In [2]: %timeit for x in City.objects.values_list('id'): pass
77.7 ms ± 1.49 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [3]: %timeit for x in City.objects.values_list('id', 'id'): pass
95.9 ms ± 1.71 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
[https://github.com/django/django/pull/8835 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/28459>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"2d136ede8abd6876b33607eea094e6903eb77d77" 2d136ede]:
{{{
#!CommitTicketReference repository=""
revision="2d136ede8abd6876b33607eea094e6903eb77d77"
Fixed #28459 -- Improved performance of ValuesListIterable.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:1>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8843 PR (Improved performance of
Model.from_db() when fields are deferred)]
Before:
{{{
In [3]: %timeit for x in City.objects.only('id'): pass
1.07 s ± 10.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
After:
{{{
In [2]: %timeit for x in City.objects.only('id'): pass
961 ms ± 5.68 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:2>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8845 PR (Improved performance of
select_related() when model is prefetched from its parent.)]
Before:
{{{
In [2]: %timeit for x in Human.objects.select_related('person'): pass
196 ms ± 2.05 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
In [2]: %timeit for x in Human.objects.select_related('person'): pass
186 ms ± 1.59 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:3>
Comment (by Tim Graham <timograham@…>):
In [changeset:"78f7f6802175c6ef9a1c5160bbad0b9d2661f603" 78f7f68]:
{{{
#!CommitTicketReference repository=""
revision="78f7f6802175c6ef9a1c5160bbad0b9d2661f603"
Refs #28459 -- Improved performance of Model.from_db() when fields are
deferred.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:4>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8849 PR (Improved performance of
SQLCompiler.results_iter())]
Before:
{{{
In [3]: %timeit for x in City.objects.values_list('id'): pass
74 ms ± 1.45 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [2]: %timeit for x in City.objects.annotate(v=models.Value(1,
output_field=models.IntegerField())).values_list('v'): pass
343 ms ± 2.62 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
After:
{{{
In [4]: %timeit for x in City.objects.values_list('id'): pass
69.1 ms ± 744 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [3]: %timeit for x in City.objects.annotate(v=models.Value(1,
output_field=models.IntegerField())).values_list('v'): pass
254 ms ± 1.61 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:5>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8851 PR (Optimized ModelState
instantiation)]
Before:
{{{
In [5]: %timeit Skill()
6.05 µs ± 54.2 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [4]: %timeit for x in Skill.objects.all(): pass
156 ms ± 1.28 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
In [2]: %timeit Skill()
5.78 µs ± 29.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [3]: %timeit for x in Skill.objects.all(): pass
149 ms ± 1.62 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:6>
Comment (by Josh Smeaton):
These are awesome! We need more of these kind of patches.
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:7>
Comment (by Tim Graham <timograham@…>):
In [changeset:"ca46f4688cd609ec3a056fc4c66fb17176a9d712" ca46f468]:
{{{
#!CommitTicketReference repository=""
revision="ca46f4688cd609ec3a056fc4c66fb17176a9d712"
Refs #28459 -- Improved performance of SQLCompiler.results_iter().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:8>
Comment (by Tim Graham <timograham@…>):
In [changeset:"ba1d82df63a5cc49c4a472fc57acdb735c6fb66c" ba1d82df]:
{{{
#!CommitTicketReference repository=""
revision="ba1d82df63a5cc49c4a472fc57acdb735c6fb66c"
Refs #28459 -- Improved performance of select_related() when model is
prefetched from its parent.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:9>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8867 PR (Improved performance of
ValuesIterable)]
Before:
{{{
In [6]: %timeit -n 10 for x in City.objects.values('id', 'b'): pass
212 ms ± 4.3 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
In [5]: %timeit for x in City.objects.values('id', 'b'): pass
151 ms ± 850 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:10>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8854 PR (Improved performance of
BaseExpression.convert_value())]
Before:
{{{
In [2]: %timeit for x in City.objects.annotate(v=models.Value(1,
output_field=models.IntegerField())).values_list('v'): pass
262 ms ± 3.42 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [3]: %timeit for x in City.objects.annotate(v=models.Value('1',
output_field=models.TextField())).values_list('v'): pass
243 ms ± 3.77 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
After:
{{{
In [2]: %timeit for x in City.objects.annotate(v=models.Value(1,
output_field=models.IntegerField())).values_list('v'): pass
191 ms ± 2.08 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [3]: %timeit for x in City.objects.annotate(v=models.Value('1',
output_field=models.TextField())).values_list('v'): pass
65.6 ms ± 878 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:11>
Comment (by Sergey Fedoseev <fedoseev.sergey@…>):
In [changeset:"fb0bdff981deac05388620d5670612eb96183bba" fb0bdff]:
{{{
#!CommitTicketReference repository=""
revision="fb0bdff981deac05388620d5670612eb96183bba"
Refs #28459 -- Improved performance of ValuesIterable.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:12>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8871 PR (Improved performance of
sql.compiler.cursor_iter())]
Before:
{{{
In [6]: %timeit for x in City.objects.values_list('id'): pass
67.2 ms ± 1.03 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
In [6]: %timeit for x in City.objects.values_list('id'): pass
58 ms ± 580 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:13>
Comment (by Tim Graham <timograham@…>):
In [changeset:"b8731c3fff3f36ed96655e74ab1c86263eba0119" b8731c3]:
{{{
#!CommitTicketReference repository=""
revision="b8731c3fff3f36ed96655e74ab1c86263eba0119"
Refs #28459 -- Improved performance of sql.compiler.cursor_iter().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:14>
Comment (by Tim Graham <timograham@…>):
In [changeset:"22ff86ec52bb536ad9d1fa3ee0c9a3d5eb9925c1" 22ff86e]:
{{{
#!CommitTicketReference repository=""
revision="22ff86ec52bb536ad9d1fa3ee0c9a3d5eb9925c1"
Refs #28459 -- Made Oracle get_db_converters() return converter for empty
strings only when it's needed.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:15>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8878 PR (Improved performance of
loading DecimalField on SQLite)]
Before:
{{{
In [5]: %timeit for x in Test.objects.values_list('decimal'): pass
150 ms ± 1.76 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [6]: %timeit for x in Test.objects.values_list(models.F('decimal') +
1): pass
123 ms ± 1.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
In [2]: %timeit for x in Test.objects.values_list('decimal'): pass
28.8 ms ± 1.05 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [3]: %timeit for x in Test.objects.values_list(models.F('decimal') +
1): pass
84.6 ms ± 541 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:16>
Comment (by Tim Graham <timograham@…>):
In [changeset:"a146b65628e702a9a3ed5be21542ca45366fbb29" a146b656]:
{{{
#!CommitTicketReference repository=""
revision="a146b65628e702a9a3ed5be21542ca45366fbb29"
Refs #28459 -- Improved performance of loading DecimalField on SQLite.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:17>
Comment (by Tim Graham <timograham@…>):
In [changeset:"5cc746206726c538c36a2830e7c068f1c8a0e7c8" 5cc74620]:
{{{
#!CommitTicketReference repository=""
revision="5cc746206726c538c36a2830e7c068f1c8a0e7c8"
Refs #28459 -- Optimized ModelState instantiation.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:18>
Comment (by Tim Graham <timograham@…>):
In [changeset:"47ccefeada926ffbccaa354dec7a987a7e7ca701" 47ccefea]:
{{{
#!CommitTicketReference repository=""
revision="47ccefeada926ffbccaa354dec7a987a7e7ca701"
Refs #28459 -- Improved performance of BaseExpression.convert_value().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:19>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8934 PR (Improved performance of
FlatValuesListIterable)]
Before:
{{{
In [3]: %timeit -n 100 for x in City.objects.values_list('id', flat=True):
pass
61.2 ms ± 719 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
}}}
After:
{{{
In [30]: %timeit -n 100 for x in City.objects.values_list('id',
flat=True): pass
58.8 ms ± 514 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:20>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/8935 PR (Improved performance of
SQLCompiler.apply_converters())]
Before:
{{{
In [3]: %timeit for x in City.objects.annotate(v=models.Value(1,
output_field=models.IntegerField())).values_list('v'): pass
176 ms ± 725 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [6]: %timeit for x in City.objects.annotate(v=models.Value(1,
output_field=models.IntegerField())).values('v'): pass
257 ms ± 2.18 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
After:
{{{
In [3]: %timeit for x in City.objects.annotate(v=models.Value(1,
output_field=models.IntegerField())).values_list('v'): pass
170 ms ± 1.25 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [5]: %timeit for x in City.objects.annotate(v=models.Value(1,
output_field=models.IntegerField())).values('v'): pass
224 ms ± 1.88 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:21>
Comment (by Tim Graham <timograham@…>):
In [changeset:"4dfd6b88d520b43b6363946e5ee58ba14cd1efe6" 4dfd6b88]:
{{{
#!CommitTicketReference repository=""
revision="4dfd6b88d520b43b6363946e5ee58ba14cd1efe6"
Refs #28459 -- Improved performance of FlatValuesListIterable.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:22>
Comment (by Tim Graham <timograham@…>):
In [changeset:"dca67bb2c27cfa1b23cb6519c544ce467813e0ec" dca67bb]:
{{{
#!CommitTicketReference repository=""
revision="dca67bb2c27cfa1b23cb6519c544ce467813e0ec"
Refs #28459 -- Improved performance of SQLCompiler.apply_converters().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:23>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/9070 PR (Improved performance of
loading models from DB)]
{{{
class TestModel(models.Model):
field1 = models.IntegerField(null=True)
field2 = models.IntegerField(null=True)
field3 = models.IntegerField(null=True)
field4 = models.IntegerField(null=True)
field5 = models.IntegerField(null=True)
}}}
`TestModel.objects.count() == 20000`, values for all fields `is None`.
Before:
{{{
In [7]: %timeit for x in TestModel.objects.all(): pass
163 ms ± 1.41 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
In [7]: %timeit for x in TestModel.objects.all(): pass
143 ms ± 1.07 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:24>
* status: closed => new
* resolution: fixed =>
Comment:
There's a patch so reopening the card.
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:25>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/9486 PR]
Before:
{{{
In [2]: %timeit for x in DecimalModel.objects.values_list('d'): pass
128 ms ± 531 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [3]: %timeit for x in DecimalModel.objects.values_list(models.F('d') +
1): pass
39.5 ms ± 542 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
In [2]: %timeit for x in DecimalModel.objects.values_list('d'): pass
31.8 ms ± 490 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [3]: %timeit for x in DecimalModel.objects.values_list(models.F('d') +
1): pass
20.3 ms ± 579 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:26>
Comment (by Tim Graham <timograham@…>):
In [changeset:"d0f569b350bca89eeb186523d8905a6e31b5a947" d0f569b3]:
{{{
#!CommitTicketReference repository=""
revision="d0f569b350bca89eeb186523d8905a6e31b5a947"
Refs #28459 -- Improved performance of loading DecimalField on SQLite.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:27>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/9494 PR (Improved performance of
loading DurationField on SQLite and MySQL.)]
Before:
{{{
%timeit for x in DurationModel.objects.values_list('d'): pass
168 ms ± 678 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
%timeit for x in DurationModel.objects.values_list('d'): pass
23.1 ms ± 243 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:28>
Comment (by Tim Graham <timograham@…>):
In [changeset:"ae1baa7d1d7fcb823e71ce9cb7c17de47ab2ff5e" ae1baa7d]:
{{{
#!CommitTicketReference repository=""
revision="ae1baa7d1d7fcb823e71ce9cb7c17de47ab2ff5e"
Refs #28459 -- Improved performance of loading DurationField on SQLite and
MySQL.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:29>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/9509 PR (Used default date
converter on SQLite for better performance)]
Before:
{{{
%timeit for x in TestModel.objects.values_list('date'): pass
83.3 ms ± 829 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
%timeit for x in TestModel.objects.values_list('date'): pass
40.7 ms ± 613 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:30>
Comment (by Sergey Fedoseev <fedoseev.sergey@…>):
In [changeset:"51ae4e1f324afcf1c7f6a102335dc78bfa51c126" 51ae4e1f]:
{{{
#!CommitTicketReference repository=""
revision="51ae4e1f324afcf1c7f6a102335dc78bfa51c126"
Refs #28459 -- Used default date converter on SQLite for better
performance.
See https://docs.python.org/3/library/sqlite3.html#default-adapters-and-
converters.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:31>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/9542 PR (Improved performance of
duration expressions on SQLite)]
Before:
{{{
%timeit for x in
Date.objects.annotate(d=models.ExpressionWrapper(models.F('date') +
models.F('duration'), models.DateTimeField())).values_list('d'): pass
591 ms ± 21.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
After:
{{{
%timeit for x in
Date.objects.annotate(d=models.ExpressionWrapper(models.F('date') +
models.F('duration'), models.DateTimeField())).values_list('d'): pass
390 ms ± 2.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:32>
Comment (by Tim Graham <timograham@…>):
In [changeset:"2115be616b10f4542df7155e3a4c3f02f97cc2a3" 2115be6]:
{{{
#!CommitTicketReference repository=""
revision="2115be616b10f4542df7155e3a4c3f02f97cc2a3"
Refs #28459 -- Improved performance of duration expressions on SQLite.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:33>
Comment (by Sergey Fedoseev):
[https://github.com/django/django/pull/9739 PR (Improved performance of
time difference expressions on MySQL)]
Before:
{{{
%timeit
bool(TestModel.objects.values_list(models.ExpressionWrapper(models.F('time')
- models.F('time'), output_field=models.DurationField())))
35.4 ms ± 901 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
After:
{{{
%timeit
bool(TestModel.objects.values_list(models.ExpressionWrapper(models.F('time')
- models.F('time'), output_field=models.DurationField())))
30 ms ± 810 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:34>
Comment (by Sergey Fedoseev <fedoseev.sergey@…>):
In [changeset:"ba4a9862407c8e8574c60b9f911f6b09aff1fa12" ba4a9862]:
{{{
#!CommitTicketReference repository=""
revision="ba4a9862407c8e8574c60b9f911f6b09aff1fa12"
Refs #28459 -- Improved performance of time difference expressions on
MySQL.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:35>
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:36>
* status: new => closed
* resolution: => fixed
Comment:
I think we can close this one as fixed now. There is nothing actionable
here and separate tickets can be created if further performance
improvement opportunities are identified.
--
Ticket URL: <https://code.djangoproject.com/ticket/28459#comment:37>