Re: [Django] #34881: Model-rename migration fails with IntegrityError if m2m relations to self exist
Django
exist
-------------------------------+--------------------------------------
Reporter: dennisvang | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 4.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Description changed by dennisvang:
Old description:
> == Description
>
> Please consider the (contrived) minimal example below, part of an app
> called `myapp`:
>
> {{{
> class Person(models.Model):
> name = models.CharField(max_length=255)
> children = models.ManyToManyField(
> to='self', through='myapp.Relation', blank=True
> )
>
> class Relation(models.Model):
> parent = models.ForeignKey(
> to='myapp.Person',
> related_name='relations_as_parent',
> on_delete=models.CASCADE,
> )
> child = models.ForeignKey(
> to='myapp.Person',
> related_name='relations_as_child',
> on_delete=models.CASCADE,
> )
> }}}
>
> Now suppose I rename the `Person` model to `Foo` and update corresponding
> references.
>
> Then I run `manage.py makemigrations`, which correctly recognizes that
> the model has been renamed.
>
> Now, applying this migration to an empty database works, without issue,
> but applying the migration to a database with existing data fails with an
> `IntegrityError`.
>
> == Steps to reproduce
>
> 1. start a new project, start a new app called `myapp`, with models as
> above.
> 2. run `makemigrations` and `migrate`
> 2. Load (valid) data from the following fixture:
> {{{
> [
> {"model": "myapp.person", "pk": 1, "fields": {"name": "Jenny"}},
> {"model": "myapp.person", "pk": 2, "fields": {"name": "Johnny"}},
> {"model": "myapp.person", "pk": 3, "fields": {"name": "Mom"}},
> {"model": "myapp.person", "pk": 4, "fields": {"name": "Dad"}},
> {"model": "myapp.relation", "pk": 1, "fields": {"parent": 3, "child":
> 1}},
> {"model": "myapp.relation", "pk": 2, "fields": {"parent": 3, "child":
> 2}},
> {"model": "myapp.relation", "pk": 3, "fields": {"parent": 4, "child":
> 1}},
> {"model": "myapp.relation", "pk": 4, "fields": {"parent": 4, "child":
> 2}}
> ]
> }}}
> 4. rename the `Person` model to e.g. `Foo` and update all references in
> code
> 5. run `makemigrations` and `migrate` again
>
> == What happens
>
> The `migrate` command fails with
>
> {{{
> django.db.utils.IntegrityError: The row in table 'myapp_relation' with
> primary key '1' has an invalid foreign key: myapp_relation.child_id
> contains a value '1' that does not have a corresponding value in
> myapp_person.id.
> }}}
>
> But a `Person` with `id=1` does exist in the database.
>
> == What I would expect to happen
>
> I would expect this to work without any problems.
New description:
== Description
Please consider the (contrived) minimal example below, part of an app
called `myapp`:
{{{
class Person(models.Model):
name = models.CharField(max_length=255)
children = models.ManyToManyField(
to='self', through='myapp.Relation', blank=True
)
class Relation(models.Model):
parent = models.ForeignKey(
to='myapp.Person',
related_name='relations_as_parent',
on_delete=models.CASCADE,
)
child = models.ForeignKey(
to='myapp.Person',
related_name='relations_as_child',
on_delete=models.CASCADE,
)
}}}
Now suppose I rename the `Person` model to `Foo` and update corresponding
references.
Then I run `manage.py makemigrations`, which correctly recognizes that the
model has been renamed.
Now, applying this migration to an empty database works, without issue,
but applying the migration to a database with existing data fails with an
`IntegrityError`.
== Steps to reproduce
1. start a new project, start a new app called `myapp`, with models as
above.
2. run `makemigrations` and `migrate`
2. Load (valid) data from the following fixture:
{{{
[
{"model": "myapp.person", "pk": 1, "fields": {"name": "Jenny"}},
{"model": "myapp.person", "pk": 2, "fields": {"name": "Johnny"}},
{"model": "myapp.person", "pk": 3, "fields": {"name": "Mom"}},
{"model": "myapp.person", "pk": 4, "fields": {"name": "Dad"}},
{"model": "myapp.relation", "pk": 1, "fields": {"parent": 3, "child":
1}},
{"model": "myapp.relation", "pk": 2, "fields": {"parent": 3, "child":
2}},
{"model": "myapp.relation", "pk": 3, "fields": {"parent": 4, "child":
1}},
{"model": "myapp.relation", "pk": 4, "fields": {"parent": 4, "child":
2}}
]
}}}
4. rename the `Person` model to e.g. `Foo` and update all references in
code
5. run `makemigrations` and `migrate` again
== What happens
The `migrate` command fails with
{{{
django.db.utils.IntegrityError: The row in table 'myapp_relation' with
primary key '1' has an invalid foreign key: myapp_relation.child_id
contains a value '1' that does not have a corresponding value in
myapp_person.id.
}}}
But a `Person` with `id=1` does exist in the database.
== What I would expect to happen
I would expect this to work without any problems.
== Notes
I also tried the same steps with an *implicit* `through` model , i.e.
`children = models.ManyToManyField(to='self', blank=True)`.
This works without issue.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34881#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Old description:
New description:
== Description
== Steps to reproduce
== What happens
== Notes
I also tried the same steps with an ''implicit'' `through` model , i.e.
`children = models.ManyToManyField(to='self', blank=True)`.
This works without issue.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34881#comment:5>
Django
Old description:
> I also tried the same steps with an ''implicit'' `through` model , i.e.
> `children = models.ManyToManyField(to='self', blank=True)`.
> This works without issue.
New description:
== Description
Please consider the (contrived) minimal example below, part of an app
called `myapp`:
{{{
class Person(models.Model):
name = models.CharField(max_length=255)
relatives = models.ManyToManyField(
== Steps to reproduce
== What happens
== Notes
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34881#comment:6>
Django
exist
-------------------------------+--------------------------------------
Reporter: dennisvang | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 4.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Description changed by dennisvang:
Old description:
> == Description
>
> Please consider the (contrived) minimal example below, part of an app
> called `myapp`:
>
> {{{
> class Person(models.Model):
> name = models.CharField(max_length=255)
New description:
== Description
parents_or_children = models.ManyToManyField(
== Steps to reproduce
== What happens
== Notes
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34881#comment:7>
Django
exist
-------------------------------+--------------------------------------
Reporter: dennisvang | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 4.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Comment (by Natalia Bidart):
Hello! Thank you for your ticket.
Could you please attach the migrations generated by each of the
`makemigrations` runs?
--
Ticket URL: <https://code.djangoproject.com/ticket/34881#comment:8>
Django
exist
-------------------------------+--------------------------------------
Reporter: dennisvang | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 4.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Comment (by Natalia Bidart):
Following the ticket description, I have been able to reproduce.
Migrations are attached, and SQLs are:
* 0001
{{{#!sql
BEGIN;
--
-- Create model Person
--
CREATE TABLE "ticket_34881_person" ("id" integer NOT NULL PRIMARY KEY
AUTOINCREMENT, "name" varchar(255) NOT NULL);
--
-- Create model Relation
--
CREATE TABLE "ticket_34881_relation" ("id" integer NOT NULL PRIMARY KEY
AUTOINCREMENT, "child_id" bigint NOT NULL REFERENCES "ticket_34881_person"
("id") DEFERRABLE INITIALLY DEFERRED, "parent_id" bigint NOT NULL
REFERENCES "ticket_34881_person" ("id") DEFERRABLE INITIALLY DEFERRED);
--
-- Add field parents_or_children to person
--
CREATE TABLE "new__ticket_34881_person" ("id" integer NOT NULL PRIMARY KEY
AUTOINCREMENT, "name" varchar(255) NOT NULL);
INSERT INTO "new__ticket_34881_person" ("id", "name") SELECT "id", "name"
FROM "ticket_34881_person";
DROP TABLE "ticket_34881_person";
ALTER TABLE "new__ticket_34881_person" RENAME TO "ticket_34881_person";
CREATE INDEX "ticket_34881_relation_child_id_77021f7a" ON
"ticket_34881_relation" ("child_id");
CREATE INDEX "ticket_34881_relation_parent_id_20447c41" ON
"ticket_34881_relation" ("parent_id");
COMMIT;
}}}
* 0002 (I renamed `Person` to `PersonFoo`)
{{{#!sql
BEGIN;
--
-- Rename model Person to PersonFoo
--
ALTER TABLE "ticket_34881_person" RENAME TO "ticket_34881_personfoo";
CREATE TABLE "new__ticket_34881_relation" ("id" integer NOT NULL PRIMARY
KEY AUTOINCREMENT, "child_id" bigint NOT NULL REFERENCES
"ticket_34881_personfoo" ("id") DEFERRABLE INITIALLY DEFERRED, "parent_id"
bigint NOT NULL REFERENCES "ticket_34881_person" ("id") DEFERRABLE
INITIALLY DEFERRED);
INSERT INTO "new__ticket_34881_relation" ("id", "parent_id", "child_id")
SELECT "id", "parent_id", "child_id" FROM "ticket_34881_relation";
DROP TABLE "ticket_34881_relation";
ALTER TABLE "new__ticket_34881_relation" RENAME TO
"ticket_34881_relation";
CREATE INDEX "ticket_34881_relation_child_id_77021f7a" ON
"ticket_34881_relation" ("child_id");
CREATE INDEX "ticket_34881_relation_parent_id_20447c41" ON
"ticket_34881_relation" ("parent_id");
CREATE TABLE "new__ticket_34881_relation" ("id" integer NOT NULL PRIMARY
KEY AUTOINCREMENT, "parent_id" bigint NOT NULL REFERENCES
"ticket_34881_personfoo" ("id") DEFERRABLE INITIALLY DEFERRED, "child_id"
bigint NOT NULL REFERENCES "ticket_34881_person" ("id") DEFERRABLE
INITIALLY DEFERRED);
INSERT INTO "new__ticket_34881_relation" ("id", "child_id", "parent_id")
SELECT "id", "child_id", "parent_id" FROM "ticket_34881_relation";
DROP TABLE "ticket_34881_relation";
ALTER TABLE "new__ticket_34881_relation" RENAME TO
"ticket_34881_relation";
CREATE INDEX "ticket_34881_relation_parent_id_20447c41" ON
"ticket_34881_relation" ("parent_id");
CREATE INDEX "ticket_34881_relation_child_id_77021f7a" ON
"ticket_34881_relation" ("child_id");
COMMIT;
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34881#comment:9>
Django
exist
-------------------------------+--------------------------------------
Reporter: dennisvang | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 4.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
* Attachment "0001_initial.py" added.
--
Ticket URL: <https://code.djangoproject.com/ticket/34881>
Django
exist
-------------------------------+--------------------------------------
Reporter: dennisvang | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 4.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Changes (by Natalia Bidart):
* Attachment "0002_rename_person_personfoo.py" added.
Django
exist
----------------------------+--------------------------------------
Reporter: dennisvang | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+--------------------------------------
Changes (by Natalia Bidart):
* cc: Simon Charette, Mariusz Felisiak (added)
* version: 4.2 => dev
* component: Uncategorized => Migrations
Comment:
Assuming a simpler model for `Person` where the M2M is implicit, and doing
the rename, does indeed work. The SQL for the migration is:
* 0003 (renamed `SimplerPerson` to `SimplerPersonFoo`)
{{{#!sql
BEGIN;
--
-- Rename model SimplerPerson to SimplerPersonFoo
--
ALTER TABLE "ticket_34881_simplerperson" RENAME TO
"ticket_34881_simplerpersonfoo";
CREATE TABLE "ticket_34881_simplerpersonfoo_parents_or_children" ("id"
integer NOT NULL PRIMARY KEY AUTOINCREMENT, "from_simplerpersonfoo_id"
bigint NOT NULL REFERENCES "ticket_34881_simplerpersonfoo" ("id")
DEFERRABLE INITIALLY DEFERRED, "to_simplerpersonfoo_id" bigint NOT NULL
REFERENCES "ticket_34881_simplerpersonfoo" ("id") DEFERRABLE INITIALLY
DEFERRED);
INSERT INTO "ticket_34881_simplerpersonfoo_parents_or_children" (id,
from_simplerpersonfoo_id, to_simplerpersonfoo_id) SELECT id,
from_simplerperson_id, to_simplerperson_id FROM
"ticket_34881_simplerperson_parents_or_children";
DROP TABLE "ticket_34881_simplerperson_parents_or_children";
CREATE UNIQUE INDEX
"ticket_34881_simplerpersonfoo_parents_or_children_from_simplerpersonfoo_id_to_simplerpersonfoo_id_f05f0b12_uniq"
ON "ticket_34881_simplerpersonfoo_parents_or_children"
("from_simplerpersonfoo_id", "to_simplerpersonfoo_id");
CREATE INDEX
"ticket_34881_simplerpersonfoo_parents_or_children_from_simplerpersonfoo_id_6d3cdfb4"
ON "ticket_34881_simplerpersonfoo_parents_or_children"
("from_simplerpersonfoo_id");
CREATE INDEX
"ticket_34881_simplerpersonfoo_parents_or_children_to_simplerpersonfoo_id_83aff647"
ON "ticket_34881_simplerpersonfoo_parents_or_children"
("to_simplerpersonfoo_id");
COMMIT;
}}}
It's worth noting that the content of the explicit M2M (`Relation`) has
these rows:
{{{
sqlite> SELECT * FROM ticket_34881_relation;
1|1|3
2|2|3
3|1|4
4|2|4
}}}
While the rows for the implicit one include:
{{{
sqlite> SELECT * FROM ticket_34881_simplerperson_parents_or_children;
1|3|1
2|3|2
3|1|3
4|2|3
5|4|1
6|4|2
7|1|4
8|2|4
}}}
It may look like a valid issue but I'll cc Simon and Mariusz for a second
opinion.
--
Ticket URL: <https://code.djangoproject.com/ticket/34881#comment:10>