[Django] #33462: intermittent migration error
Django
--------------------------------------+------------------------
Reporter: bcail | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: dev
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
--------------------------------------+------------------------
I ran into an intermittent migration issue on Django 4.x and PostgreSQL.
The migrations work fine on Django 3.2.x, but intermittently fail on
Django 4.x with the following exception:
django.db.utils.ProgrammingError: constraint
"polls_questioncontribution_base_question_id_25bfb2a8_fk" for relation
"polls_questioncontribution" already exists
I made a small demo repo to show the problem:
https://github.com/bcail/migration_error. I put the sqlmigrate output
from Django 3.2 and Django 4 in the repo, and it looks like Django 4
(sometimes) creates duplicate SQL statements.
I also ran git bisect on the Django code, and it looks like the problem
starts showing up with this commit:
https://github.com/django/django/commit/3d9040a50b160f8b4bb580e09f4120d4979fe29e.
--
Ticket URL: <https://code.djangoproject.com/ticket/33462>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
----------------------------+------------------------------------
Reporter: bcail | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: dev
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Pedro Schlickmann Mendes):
* stage: Unreviewed => Accepted
Comment:
Succesfully reproduced it. But I'm still trying to figure it out why this
happens only sometimes for you. For me it happens everytime I try to run
the 0008 migration for the first time, the second time it runs perfectly.
The same happens when I try to revert it to the 0004 migration, the first
time fails, and the second time it runs perfectly
--
Ticket URL: <https://code.djangoproject.com/ticket/33462#comment:1>
Django
----------------------------+------------------------------------
Type: Bug | Status: new
Component: Migrations | Version: dev
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by bcail):
Using "python manage.py test -v 2" to run the migrations on a test db, it
fails for me pretty often. However, it did succeed after I restarted
postgresql (and then it failed again). Another time after I restarted
postgresql, it failed the first time.
--
Ticket URL: <https://code.djangoproject.com/ticket/33462#comment:2>
Django
operations.
---------------------------------+------------------------------------
Reporter: bcail | Owner: nobody
Type: Bug | Status: new
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* cc: David Wobrock (added)
* version: dev => 4.0
* severity: Normal => Release blocker
Comment:
Thanks for the report! This is caused by multiple references to the same
table (MTI and foreign keys). I was able to fix this by excluding
relations that were already yielded:
{{{#!diff
diff --git a/django/db/backends/base/schema.py
b/django/db/backends/base/schema.py
index 47fdae8fd4..c057092564 100644
--- a/django/db/backends/base/schema.py
+++ b/django/db/backends/base/schema.py
@@ -33,19 +33,22 @@ def _all_related_fields(model):
return model._meta._get_fields(forward=False, reverse=True,
include_hidden=True)
-def _related_non_m2m_objects(old_field, new_field):
+def _related_non_m2m_objects(old_field, new_field, excluded=None):
# Filter out m2m objects from reverse relations.
# Return (old_relation, new_relation) tuples.
related_fields = zip(
(obj for obj in _all_related_fields(old_field.model) if
_is_relevant_relation(obj, old_field)),
(obj for obj in _all_related_fields(new_field.model) if
_is_relevant_relation(obj, new_field)),
)
+ excluded = excluded or ()
for old_rel, new_rel in related_fields:
- yield old_rel, new_rel
- yield from _related_non_m2m_objects(
- old_rel.remote_field,
- new_rel.remote_field,
- )
+ if (old_rel, new_rel) not in excluded:
+ yield old_rel, new_rel
+ yield from _related_non_m2m_objects(
+ old_rel.remote_field,
+ new_rel.remote_field,
+ excluded=(*related_fields, *excluded),
+ )
class BaseDatabaseSchemaEditor:
}}}
I'm not 100% satisfied with this patch.
This is really a regression in 325d7710ce9f6155bb55610ad6b4580d31263557.
--
Ticket URL: <https://code.djangoproject.com/ticket/33462#comment:3>
Django
operations.
Reporter: bcail | Owner: Mariusz
| Felisiak
Type: Bug | Status: assigned
Component: Migrations | Version: 4.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* owner: nobody => Mariusz Felisiak
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/33462#comment:4>
Django
operations.
-------------------------------------+-------------------------------------
Reporter: bcail | Owner: Mariusz
| Felisiak
Type: Bug | Status: assigned
Component: Migrations | Version: 4.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* has_patch: 0 => 1
Comment:
I found an easier solution, [https://github.com/django/django/pull/15364
PR].
--
Ticket URL: <https://code.djangoproject.com/ticket/33462#comment:5>
Django
operations.
-------------------------------------+-------------------------------------
Reporter: bcail | Owner: Mariusz
| Felisiak
Component: Migrations | Version: 4.0
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by GitHub <noreply@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"e972620ada4f9ed7bc57f28e133e85c85b0a7b20" e972620a]:
{{{
#!CommitTicketReference repository=""
revision="e972620ada4f9ed7bc57f28e133e85c85b0a7b20"
Fixed #33462 -- Fixed migration crash when altering type of primary key
with MTI and foreign key.
This prevents duplicated operations when altering type of primary key
with MTI and foreign key. Previously, a foreign key to the base model
was added twice, once directly and once by the inheritance model.
Thanks bcail for the report.
Regression in 325d7710ce9f6155bb55610ad6b4580d31263557.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33462#comment:6>
Django
operations.
-------------------------------------+-------------------------------------
Reporter: bcail | Owner: Mariusz
| Felisiak
Type: Bug | Status: closed
Component: Migrations | Version: 4.0
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"7c2d4d943b2905a8cfbb2d2f6c6fa15983b23ff1" 7c2d4d94]:
{{{
#!CommitTicketReference repository=""
revision="7c2d4d943b2905a8cfbb2d2f6c6fa15983b23ff1"
[4.0.x] Fixed #33462 -- Fixed migration crash when altering type of
primary key with MTI and foreign key.
This prevents duplicated operations when altering type of primary key
with MTI and foreign key. Previously, a foreign key to the base model
was added twice, once directly and once by the inheritance model.
Thanks bcail for the report.
Regression in 325d7710ce9f6155bb55610ad6b4580d31263557.
Backport of e972620ada4f9ed7bc57f28e133e85c85b0a7b20 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33462#comment:7>