[Django] #30138: Return pk of created objects when ignore_conflicts set True on QuerySet.bulk_create()

[Django]

#30138: Return pk of created objects when ignore_conflicts set True on
QuerySet.bulk_create()
-------------------------------------+-------------------------------------
Reporter: saber- | Owner: nobody
solooki |
Type: | Status: new
Uncategorized |
Component: Database | Version: 2.2
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
According to [https://code.djangoproject.com/ticket/28668#comment:24]
comment, I create new ticket for returning pk of created objects. I tried
it in Postgresql 10.6 and this subject is supported. I see that when
ignore_confilicts set as True, django api doesn't put ' RETURNING
"mymodel"."id" ' in query.

--
Ticket URL: <https://code.djangoproject.com/ticket/30138>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------
Reporter: saber-solooki | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Tim Graham):

* type: Uncategorized => Cleanup/optimization
* stage: Unreviewed => Accepted

Old description:

> According to [https://code.djangoproject.com/ticket/28668#comment:24]
> comment, I create new ticket for returning pk of created objects. I tried
> it in Postgresql 10.6 and this subject is supported. I see that when
> ignore_confilicts set as True, django api doesn't put ' RETURNING
> "mymodel"."id" ' in query.

New description:

As requested in ticket:28668#comment:24, here's a ticket for setting the
primary keys of created objects. I tried it in PostgreSQL 10.6 and this
could be supported. When `ignore_confilicts=True`, Django doesn't put `'
RETURNING "mymodel"."id" '` in the query.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:1>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------
Reporter: saber solooki | Owner: Seunghun
Type: | Lee
Cleanup/optimization | Status: assigned

Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Seunghun Lee):

* status: new => assigned
* owner: nobody => Seunghun Lee

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:2>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------
Reporter: saber solooki | Owner: Seunghun
Type: | Lee
Cleanup/optimization | Status: assigned
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Дилян Палаузов):

At https://code.djangoproject.com/ticket/28668 are attached patches, that
accomplish this.

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:3>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------
Reporter: saber solooki | Owner: Seunghun
Type: | Lee
Cleanup/optimization | Status: assigned
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Tim Graham):

* needs_better_patch: 0 => 1
* has_patch: 0 => 1

Comment:

[https://github.com/django/django/pull/10998 PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:4>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------

Reporter: saber solooki | Owner: (none)
Type: | Status: new
Cleanup/optimization |

Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Seunghun Lee):

* status: assigned => new
* owner: Seunghun Lee => (none)

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:5>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------

Reporter: saber solooki | Owner: Hasan
Type: | Ramezani
Cleanup/optimization | Status: assigned

Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Hasan Ramezani):

* owner: (none) => Hasan Ramezani

* status: new => assigned

Comment:

Based on the ticket description we should add `' RETURNING "mymodel"."id"
'` to the query when `ignore_confilicts=True`. But based on the
[https://github.com/django/django/pull/12233#issuecomment-567965748 Simon
comment on the PR] `the database will not return any conflicting row ids`.
I am a little confused because of the Database just returns newly created
object ids. Check Simon's example in the comment.
What should we do?

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:6>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------
Reporter: saber solooki | Owner: Hasan
Type: | Ramezani
Cleanup/optimization | Status: assigned
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

Hasan, this was mentioned in
https://code.djangoproject.com/ticket/28668#comment:21 but was dismissed
as something
[https://code.djangoproject.com/attachment/ticket/28668/on_conflict_ignore.patch
the proposed implementation] would take of.

It does so by doing a `LEFT JOIN` over the existing rows and would require
a set of columns against which to join to be provided to the call.

Implementing this in a proper manager would likely require a lot of work.
It would require `ignore_conflicts` to accept an iterable of constraint
names or tuple of fields corresponding to `Field.unique=True` or
`Meta.unique_together` definitions. These entries could then be matched to
constraint names and used as `conflict_target` in the `ON CONFLICT` clause
and in the join `USING` clause. I think we'd also like to allow exclusion
constraints to be treated differently because joining on fields won't help
here as an insert could be conflicting with more than one result.

I assume callers would also like to be able to differentiate between newly
inserted and conflicting entries so the return value of `bulk_create`
would need to be formalized/changed.

In summary this ticket involves a lot of research work to get right and
not paint ourselves in a corner with regards to the desired API.

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:7>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------

Reporter: saber solooki | Owner: (none)
Type: | Status: new
Cleanup/optimization |

Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Hasan Ramezani):

* owner: Hasan Ramezani => (none)

* status: assigned => new

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:8>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when
ignore_conflicts=True
-------------------------------------+-------------------------------------
Reporter: saber solooki | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Alex Vandiver):

* cc: Alex Vandiver (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:9>

[Django]

#30138: Allow QuerySet.bulk_create() to set pk of created objects when

ignore_conflicts=True
-------------------------------------+-------------------------------------
Reporter: saber solooki | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Ülgen Sarıkavak):

* cc: Ülgen Sarıkavak (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/30138#comment:10>