[Django] #36858: Optimize `db_default` creation

[Django]

#36858: Optimize `db_default` creation
-------------------------------------+-------------------------------------
Reporter: Adam | Owner: Adam Johnson
Johnson |
Type: | Status: assigned
Cleanup/optimization |
Component: Database | Version: dev
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Currently, `Field._get_default()` returns a `lambda` that instantiates a
new `DatabaseDefault` each time it is called. This expression is identical
each time, so this work is wasted. We can optimize it by creating a single
expression that is returned each time.

Thanks to Adam Sołtysik for the hint in https://forum.djangoproject.com/t
/faster-bulk-create-using-dictionaries/43891

Benchmarked with a quick modification to the test suite:

{{{#!diff
diff --git ./tests/bulk_create/tests.py ./tests/bulk_create/tests.py
index 397fcb9186..ddfe315c0c 100644
--- ./tests/bulk_create/tests.py
+++ ./tests/bulk_create/tests.py
@@ -877,6 +877,16 @@ def test_db_default_field_excluded(self):
2 if connection.features.can_return_rows_from_bulk_insert
else 1,
)

+
+ def test_benchmark(self):
+ import time
+ start = time.perf_counter()
+ DbDefaultModel.objects.bulk_create(
+ [DbDefaultModel(name=f"obj {i}") for i in range(100_000)]
+ )
+ end = time.perf_counter()
+ print(f"{end - start:.3f} seconds")
+
@skipUnlessDBFeature(
"can_return_rows_from_bulk_insert",
"supports_expression_defaults"
)
}}}

Run with:

{{{
./runtests.py --parallel 1
bulk_create.tests.BulkCreateTests.test_benchmark -v 0
System check identified no issues (0 silenced).
0.617 seconds
----------------------------------------------------------------------
Ran 1 test in 0.618s

OK
}}}

Results, best of 3:

Before: 0.691 seconds
After: 0.610 seconds

A ~12% speedup on this `bulk_create()` operation, on a model with a single
`db_default` field.
--
Ticket URL: <https://code.djangoproject.com/ticket/36858>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36858: Optimize `db_default` creation
-------------------------------------+-------------------------------------

Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* stage: Unreviewed => Accepted

Comment:

Thanks for that Adam! When I read the thread I figured something was
definitely wrong but I didn't have the time to look at it. Could you also
submit a minimal benchmark againt django-asv that makes use of
`db_default`?
--
Ticket URL: <https://code.djangoproject.com/ticket/36858#comment:1>

[Django]

#36858: Optimize `db_default` creation
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36858#comment:2>

[Django]

#36858: Optimize `db_default` creation
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson

Cleanup/optimization | Status: closed

Component: Database layer | Version: dev
(models, ORM) |

Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls <jacobtylerwalls@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"2b192bff26cf956c168790fce6a637cbd814250b" 2b192bff]:
{{{#!CommitTicketReference repository=""
revision="2b192bff26cf956c168790fce6a637cbd814250b"
Fixed #36858 -- Optimized `Field._get_default()` for `db_default` case.

Create and share a single instance of `DatabaseDefault` instead of making
a new
one each time the lambda is called. The quick benchmark on the ticket
shows a
~12% speedup for a large `bulk_create()` operation.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36858#comment:3>

[Django]

#36858: Optimize `db_default` creation
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Adam Johnson):

Replying to [comment:1 Simon Charette]:

> Thanks for that Adam! When I read the thread I figured something was
definitely wrong but I didn't have the time to look at it. Could you also
submit a minimal benchmark againt django-asv that makes use of
`db_default`?

https://github.com/django/django-asv/pull/95
--
Ticket URL: <https://code.djangoproject.com/ticket/36858#comment:4>