[Django] #36260: bulk_create() does not fetch primary keys if they are generated by database

[Django]

#36260: bulk_create() does not fetch primary keys if they are generated by database
-------------------------------------+-------------------------------------
Reporter: Dmitry | Owner: Dmitry Shachnev
Shachnev |
Type: | Status: assigned
Uncategorized |
Component: | Version: 5.2
Uncategorized |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
In our project, we use a function that generates UUID on database side
(it’s `uuid7` function from PostgreSQL 18).

The `bulk_create` function uses `_is_pk_set()` method to check if the
object already has a primary key assigned. However, `_is_pk_set()` wrongly
returns True in this case: primary key is not None, however it is a
`DatabaseDefault` instance and does not actually return a value.

Because of this, the objects returned by `bulk_create` do not actually
have an ID.

I prepared a patch to fix this, and wrote a test case for it. Without my
patch, the test fails with the following error:
{{{
======================================================================
FAIL: test_db_default_primary_key
(bulk_create.tests.BulkCreateTests.test_db_default_primary_key)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/dmitry/upstream/django/tests/bulk_create/tests.py", line
878, in test_db_default_primary_key
self.assertIsInstance(obj1.id, UUID)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
AssertionError: <django.db.models.expressions.DatabaseDefault object at
0x7f2563f6b570> is not an instance of <class 'uuid.UUID'>

----------------------------------------------------------------------
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36260>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36260: bulk_create() does not fetch primary keys if they are generated by database
-------------------------------------+-------------------------------------

Reporter: Dmitry Shachnev | Owner: Dmitry
| Shachnev
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* component: Uncategorized => Database layer (models, ORM)
* stage: Unreviewed => Accepted
* type: Uncategorized => Bug

Comment:

Not a release blocker as `db_default` was introduced in 5.0 and I could
reproduce the issue against it.
--
Ticket URL: <https://code.djangoproject.com/ticket/36260#comment:1>

[Django]

#36260: bulk_create() does not fetch primary keys if they are generated by database
-------------------------------------+-------------------------------------
Reporter: Dmitry Shachnev | Owner: Dmitry
| Shachnev
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36260#comment:2>

[Django]

#36260: bulk_create() does not fetch primary keys if they are generated by database
-------------------------------------+-------------------------------------
Reporter: Dmitry Shachnev | Owner: Dmitry
| Shachnev
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

* needs_better_patch: 1 => 0

Comment:

Patch is LGTM. Left a comment about a potential improvement to object
partitioning by primary key that could be done in a follow up optimization
commit while we're around.
--
Ticket URL: <https://code.djangoproject.com/ticket/36260#comment:3>

[Django]

#36260: bulk_create() does not fetch primary keys if they are generated by database
-------------------------------------+-------------------------------------
Reporter: Dmitry Shachnev | Owner: Dmitry
| Shachnev
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36260#comment:4>

[Django]

#36260: bulk_create() does not fetch primary keys if they are generated by database
-------------------------------------+-------------------------------------
Reporter: Dmitry Shachnev | Owner: Dmitry
| Shachnev

Type: Bug | Status: closed

Component: Database layer | Version: 5.2
(models, ORM) |

Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"77b4ecbd53635b94d3685d16af9e8b2f96cf952c" 77b4ecb]:
{{{#!CommitTicketReference repository=""
revision="77b4ecbd53635b94d3685d16af9e8b2f96cf952c"
Fixed #36260 -- Made bulk_create() work with DB-generated primary keys.

Co-authored-by: Simon Charette <chare...@gmail.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36260#comment:5>

[Django]

#36260: bulk_create() does not fetch primary keys if they are generated by database
-------------------------------------+-------------------------------------
Reporter: Dmitry Shachnev | Owner: Dmitry
| Shachnev
Type: Bug | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"7d9aab8da0a06787d649762a702e1a518d843a63" 7d9aab8]:
{{{#!CommitTicketReference repository=""
revision="7d9aab8da0a06787d649762a702e1a518d843a63"
Refs #36260 -- Moved _is_pk_set checks into _prepare_for_bulk_create().

To avoid looping over objs twice.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36260#comment:6>

[Django]

#36260: bulk_create() does not fetch primary keys if they are generated by database
-------------------------------------+-------------------------------------
Reporter: Dmitry Shachnev | Owner: Dmitry
| Shachnev
Type: Bug | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by GitHub <noreply@…>):

In [changeset:"02e7a162a03801fce34c78d5d75fef0aaf997ab5" 02e7a16]:
{{{#!CommitTicketReference repository=""
revision="02e7a162a03801fce34c78d5d75fef0aaf997ab5"
Refs #36088, Refs #36260 - Added supports_expression_defaults checks in
bulk_create() tests.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36260#comment:7>