[Django] #36490: Avoid unnecessary transaction in QuerySet.bulk_create

[Django]

#36490: Avoid unnecessary transaction in QuerySet.bulk_create
-------------------------------------+-------------------------------------
Reporter: Simon | Owner: Simon Charette
Charette |
Type: | Status: assigned
Cleanup/optimization |
Component: Database | Version: 5.2
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
The QuerySet.bulk_create method creates an unnecessary transaction when it
can perform the insert in a singe `INSERT` when the following criteria are
met

1. There's no mix of objects with and without primary key (this could
potentially be optimized to be done in a single query but that's another
can of worm)
2. The objects fit in a single batch with respect to the provided
`batch_size` and `connection.ops.bulk_batch_size`

This is unnecessary overhead we've optimized in other areas of the code
base (e.g. `Model.save`) which seems warranted in this case particularly
because `bulk_create` offers a more featureful interface than `create`
[https://johnnymetz.com/posts/postgresql-failing-insert/ which can be used
to insert a small number of rows].
--
Ticket URL: <https://code.djangoproject.com/ticket/36490>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36490: Avoid unnecessary transaction in QuerySet.bulk_create
-------------------------------------+-------------------------------------

Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by David Sanders):

* stage: Unreviewed => Accepted

--
Ticket URL: <https://code.djangoproject.com/ticket/36490#comment:1>

[Django]

#36490: Avoid unnecessary transaction in QuerySet.bulk_create
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36490#comment:2>

[Django]

#36490: Avoid unnecessary transaction in QuerySet.bulk_create
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36490#comment:3>

[Django]

#36490: Avoid unnecessary transaction in QuerySet.bulk_create
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette

Cleanup/optimization | Status: closed

Component: Database layer | Version: 5.2
(models, ORM) |

Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"5eca562ac3cbfd3b0c6522f3d7ad4be034e1cb8c" 5eca562a]:
{{{#!CommitTicketReference repository=""
revision="5eca562ac3cbfd3b0c6522f3d7ad4be034e1cb8c"
Refs #36490 -- Simplified QuerySet._batched_insert returning fields
handling.

Whether or not returning_fields should be specified to _insert is not a
function of each batches so the conditional can be moved outside of the
loop.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36490#comment:5>

[Django]

#36490: Avoid unnecessary transaction in QuerySet.bulk_create
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"e1671278e88265e64811657b8b939b5d786295cb" e1671278]:
{{{#!CommitTicketReference repository=""
revision="e1671278e88265e64811657b8b939b5d786295cb"
Fixed #36490 -- Avoided unnecessary transaction in bulk_create.

When dealing with an heterogeneous set of object with regards to primary
key
assignment that fits in a single batch there's no need to wrap the single
INSERT statement in a transaction.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36490#comment:4>