[Django] #31301: bulk_create with ForeignKey fields with mixed filled/None values fails
Django
-------------------------------------+-------------------------------------
Reporter: Hans | Owner: nobody
Aarne Liblik |
Type: Bug | Status: new
Component: Database | Version: 3.0
layer (models, ORM) |
Severity: Normal | Keywords: oracle sql
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I have a model with ForeignKey (null=True) to another model
{{{
class A(models.Model):
name = models.CharField()
class B(models.Model):
name = models.CharField()
a = models.ForeignKey(A, on_delete=models.CASCADE, null=True)
}}}
When I try to use bulk_create as follows
{{{
B.objects.bulk_create([
B(name='first', a_id=1)
B(name='seconds', a_id=None)
])
}}}
the query fails with error
{{{
django.db.utils.DatabaseError: ORA-01790: expression must have same
datatype as corresponding expression
}}}
I got bulk_create working if the ForeignKey references are either All None
or All NOT None. It seems like the fix introduced in
https://code.djangoproject.com/ticket/22669 fixes the case for
NumberFields and CharFields, but ForeignKey field 'target_field' for me is
'AutoField' which is not defined in
{{{
django/db/backends/oracle/utils.py#52 class BulkInsertMapper:
}}}
Manually editing that this file and adding
{{{
class BulkInsertMapper:
...
types: {
...
...
'AutoField': NUMBER
}
}}}
fixes the issue for me.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: oracle sql | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):
* easy: 0 => 1
* stage: Unreviewed => Accepted
Comment:
That seems like the correct approach. Would you be interested in
submitting a Github PR that addresses the issue?
Based on c4e2fc5d9872c9a0c9c052a2e124f8a9b87de9b4 adding a nullable
foreign key on `NullableFields` to another test model of the same module
should be sufficient to augment the `test_bulk_insert_nullable_fields`
test to cover this case.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:1>
Django
-------------------------------------+-------------------------------------
| Patel
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* owner: nobody => Hardik Patel
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:2>
Django
-------------------------------------+-------------------------------------
| Patel
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Hans Aarne Liblik):
Any progress on this? If not, I can take over.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:3>
Django
-------------------------------------+-------------------------------------
| Patel
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Hardik Patel):
Replying to [comment:3 Hans Aarne Liblik]:
> Any progress on this? If not, I can take over.
It is looking harder than I expected. I am just a newbie. I will come back
later. Feel free to take over
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:4>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Hardik Patel):
* owner: Hardik Patel => (none)
* status: assigned => new
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:5>
Django
-------------------------------------+-------------------------------------
| Aarne Liblik
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* owner: (none) => Hans Aarne Liblik
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:6>
Django
-------------------------------------+-------------------------------------
| Aarne Liblik
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* cc: Marti Raudsepp (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:7>
Django
-------------------------------------+-------------------------------------
| Aarne Liblik
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by felixxm):
Hans, Please remember to add also `SmallAutoField` and `BigAutoField`.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:8>
Django
-------------------------------------+-------------------------------------
| Aarne Liblik
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
* stage: Accepted => Ready for checkin
Comment:
https://github.com/django/django/pull/12506
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:9>
Django
-------------------------------------+-------------------------------------
| Aarne Liblik
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Hans Aarne Liblik):
* stage: Ready for checkin => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:10>
Django
-------------------------------------+-------------------------------------
| Aarne Liblik
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:11>
Django
-------------------------------------+-------------------------------------
| Aarne Liblik
Type: Bug | Status: assigned
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by felixxm):
* needs_tests: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by felixxm):
* needs_better_patch: 1 => 0
* version: 3.0 => master
* needs_tests: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Keywords: oracle sql | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"a21f7b91db273a03abfb47b0580bb39e0043c99a" a21f7b91]:
{{{
#!CommitTicketReference repository=""
revision="a21f7b91db273a03abfb47b0580bb39e0043c99a"
Fixed #31301 -- Fixed crash of QuerySet.bulk_create() with mixed empty and
set ForeignKeys to AutoFields on Oracle.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: oracle sql | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Hans Aarne Liblik):
This didn't get into 3.0.4 release?
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: oracle sql | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by felixxm):
No, this doesn't qualify for a backport, see
https://docs.djangoproject.com/en/3.0/internals/release-process/ for more
details.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:16>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: oracle sql | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Marti Raudsepp):
@felixxm, are you sure this doesn't qualify?
It seems like a significant issue to me; the criteria does list "Crashing
bugs".
If I understand correctly, it causes different database backends to behave
differently? Which is always a bad surprise.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:17>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: oracle sql | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by felixxm):
It doesn't qualify for a backport from the same reason that #22669 didn't
qualify. It's not a regression it existed since
9fe050a27598cdce7f2ef7a332b4818b493a8702 (Django 1.4) so it's not crucial.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:18>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: oracle sql | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Marti Raudsepp):
What's your suggested solution for users who are hitting this bug now? I
can think of:
* Fork Django, backport the fix manually, and continue to maintain the
fork until Django 3.1.0 is released
* Upgrade to Django development "master" branch, which sometimes breaks
third-party Django addons
* Stop using Django until August 2020 rolls around (just sarcastic,
sorry!)
Am I missing something? This is behavior that everyone agrees is a bug,
and a helpful user provided pull request to fix it. Yet you are leaving
users hanging without a good solution for ~5 months.
Seems like a bug in the backport policy itself :)
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:19>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: oracle sql | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by felixxm):
I'm sorry Marti but that's our policy. We're fixing dozens of bugs in each
month and only few of them are backported. We don't backport all bugfixes
to minimize the risk of regressions.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:20>
Django
-------------------------------------+-------------------------------------
Reporter: Hans Aarne Liblik | Owner: Hans
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: oracle sql | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Claude Paroz):
Replying to [comment:19 Marti Raudsepp]:
> * Fork Django, backport the fix manually, and continue to maintain the
fork until Django 3.1.0 is released and upgrading is viable
I think that's clearly the way to go if the fix is critical to your
application. You may have to update your fork once a month if Django
periodic minor release contains security fixes, but that's a price to pay
for Django general stability where you can almost blindly apply minor
upgrades without worrying. And even with this strict backport policy, it
has happened in the past that such fixes have introduced regressions, so I
don't think we should relax this policy.
--
Ticket URL: <https://code.djangoproject.com/ticket/31301#comment:21>