[Django] #24576: Bad cascading leads to non-deterministic IntegrityError
Django
----------------------------------------------+--------------------
Reporter: glic3rinu | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: 1.8
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+--------------------
I've spent all day with a bug on my application that seems to be a bug on
Django's ORM delete on cascade resoultion order.
Moreover because the order in which related objects are deleted by the ORM
is non-deterministic (changes every time you start the interpreter) it
makes even harder to track down.
So far I've been able to reproduce the problem with a few models:
{{{
from django.db import models
from django.contrib.contenttypes.fields import GenericForeignKey,
GenericRelation
from django.contrib.contenttypes.models import ContentType
from django.db.models.signals import post_delete
from django.dispatch import receiver
class Resource(models.Model):
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey()
class Account(models.Model):
name = models.CharField(max_length=10)
resources = GenericRelation(Resource)
class Order(models.Model):
account = models.ForeignKey(Account)
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey()
class MetricStorage(models.Model):
order = models.ForeignKey(Order)
@receiver(post_delete, dispatch_uid="orders.cancel_orders")
def cancel_orders(sender, **kwargs):
instance = kwargs['instance']
print('delete', sender, instance, instance.pk)
if sender == Resource:
related = instance.content_object
if related:
ct = ContentType.objects.get_for_model(related)
order = Order.objects.get(content_type=ct,
object_id=related.pk)
order.metricstorage_set.create()
order.save()
else:
print('related is None')
}}}
And this test code
{{{
from test.models import Account, Resource, Order
account = Account.objects.create(name='John')
resource = Resource.objects.create(content_object=account)
order = Order.objects.create(account=account, content_object=account)
account.delete()
}}}
In order to properly test this you should make tries restarting the
interpreter a handfull of times, and then you'll se two different results.
This that I consider correct:
{{{
delete <class 'test.models.Order'> Order object 384
delete <class 'test.models.Account'> Account object 124
delete <class 'test.models.Resource'> Resource object 307
related is None
}}}
And the IntegrityError which I consider to be a bug
{{{
delete <class 'test.models.Resource'> Resource object 311
delete <class 'test.models.Order'> Order object 388
delete <class 'test.models.Account'> Account object 128
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python3.4/dist-packages/django/db/models/base.py",
line 872, in delete
collector.delete()
File "/usr/local/lib/python3.4/dist-
packages/django/db/models/deletion.py", line 314, in delete
sender=model, instance=obj, using=self.using
File "/usr/local/lib/python3.4/dist-packages/django/db/transaction.py",
line 232, in __exit__
connection.commit()
File "/usr/local/lib/python3.4/dist-
packages/django/db/backends/base/base.py", line 173, in commit
self._commit()
File "/usr/local/lib/python3.4/dist-
packages/django/db/backends/base/base.py", line 142, in _commit
return self.connection.commit()
File "/usr/local/lib/python3.4/dist-packages/django/db/utils.py", line
97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.4/dist-packages/django/utils/six.py", line
658, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.4/dist-
packages/django/db/backends/base/base.py", line 142, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table
"test_metricstorage" violates foreign key constraint
"test_metricstorage_order_id_730c757c66b8c627_fk_test_order_id"
DETAIL: Key (order_id)=(388) is not present in table "test_order".
}}}
Notice how the order in which related objects are deleted is differnet, I
believe this to be the source of the problem.
I've noticed this problem on 1.7 and now 1.8, not tested with master.
--
Ticket URL: <https://code.djangoproject.com/ticket/24576>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: glic3rinu | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: => 0
* needs_docs: => 0
* needs_tests: => 0
* stage: Unreviewed => Accepted
Old description:
> And this test code
>
New description:
I've spent all day with a bug on my application that seems to be a bug on
Django's ORM delete on cascade resolution order.
And this test code
interpreter a handful of times, and then you'll see two different results.
Notice how the order in which related objects are deleted is different, I
believe this to be the source of the problem.
I've noticed this problem on 1.7 and now 1.8, not tested with master.
--
Comment:
I could also reproduce on master with your example code.
--
Ticket URL: <https://code.djangoproject.com/ticket/24576#comment:1>
Django
Reporter: glic3rinu | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by sww):
Looks like this behavior's been here a while - looks like commit
272de9eb6baad45abec029aae92c2b7d9478c841 introduced the bug, so it's been
a bug since 1.6.
--
Ticket URL: <https://code.djangoproject.com/ticket/24576#comment:2>
Django
Reporter: glic3rinu | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by laurafeier):
* cc: feierlaura10@… (added)
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/5545 PR]
Tests pass under SQLite, MySQL and PostgreSQL
--
Ticket URL: <https://code.djangoproject.com/ticket/24576#comment:3>
Django
Reporter: glic3rinu | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by charettes):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/24576#comment:4>
Django
Reporter: glic3rinu | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 1.8
(models, ORM) |
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham <timograham@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"7862cbda86cdbced5fbfe4f0056105e657c1f92c" 7862cbda]:
{{{
#!CommitTicketReference repository=""
revision="7862cbda86cdbced5fbfe4f0056105e657c1f92c"
Fixed #24576 -- Made deletion of related objects deterministic.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/24576#comment:5>