[Django] #34138: Adding ManyToManyField on SQLite rebuilds table.
Django
-----------------------------------------+-------------------------
Reporter: David Wobrock | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 4.1
Severity: Normal | Keywords: sqlite3
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+-------------------------
Hey there,
While updating the [https://github.com/3YOURMIND/django-migration-linter
django-migration-linter] for Django 4.1 (yeah, a bit late to the party,
but I was busy eh :P), I bumped into what seems to be a regression.
On SQLite, when adding a ManyToManyField to a table, it seems to rebuild
the table - whereas it did not in Django 4.0.
From my understanding, rebuilding the table is not necessary in this case.
Or perhaps I'm missing something.
Steps to reproduce:
1/ Before models:
{{{
class A(models.Model):
pass
class B(models.Model):
pass
}}}
(with it's boring migration)
2/ After models:
{{{
class A(models.Model):
pass
class B(models.Model):
many_to_many = models.ManyToManyField(A)
}}}
Which, expectedly, generates the migration:
{{{
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [("app_add_manytomany_field", "0001_initial")]
operations = [
migrations.AddField(
model_name="b",
name="many_to_many",
field=models.ManyToManyField(to="app_add_manytomany_field.A"),
)
]
}}}
All good up until here.
**Now the "regression"**, in Django 4.0, a `sqlmigrate` generates:
{{{
BEGIN;
--
-- Add field many_to_many to b
--
CREATE TABLE "app_add_manytomany_field_b_many_to_many" ("id" integer NOT
NULL PRIMARY KEY AUTOINCREMENT, "b_id" integer NOT NULL REFERENCES
"app_add_manytomany_field_b" ("id") DEFERRABLE INITIALLY DEFERRED, "a_id"
integer NOT NULL REFERENCES "app_add_manytomany_field_a" ("id") DEFERRABLE
INITIALLY DEFERRED);
CREATE UNIQUE INDEX
"app_add_manytomany_field_b_many_to_many_b_id_a_id_3e15251d_uniq" ON
"app_add_manytomany_field_b_many_to_many" ("b_id", "a_id");
CREATE INDEX "app_add_manytomany_field_b_many_to_many_b_id_953b185b" ON
"app_add_manytomany_field_b_many_to_many" ("b_id");
CREATE INDEX "app_add_manytomany_field_b_many_to_many_a_id_4b44832a" ON
"app_add_manytomany_field_b_many_to_many" ("a_id");
COMMIT;
}}}
whereas in Django 4.1:
{{{
BEGIN;
--
-- Add field many_to_many to b
--
CREATE TABLE "new__app_add_manytomany_field_b" ("id" integer NOT NULL
PRIMARY KEY AUTOINCREMENT, "x" integer NOT NULL);
CREATE TABLE "app_add_manytomany_field_b_many_to_many" ("id" integer NOT
NULL PRIMARY KEY AUTOINCREMENT, "b_id" integer NOT NULL REFERENCES
"app_add_manytomany_field_b" ("id") DEFERRABLE INITIALLY DEFERRED, "a_id"
integer NOT NULL REFERENCES "app_add_manytomany_field_a" ("id") DEFERRABLE
INITIALLY DEFERRED);
INSERT INTO "new__app_add_manytomany_field_b" ("id", "x") SELECT "id", "x"
FROM "app_add_manytomany_field_b";
DROP TABLE "app_add_manytomany_field_b";
ALTER TABLE "new__app_add_manytomany_field_b" RENAME TO
"app_add_manytomany_field_b";
CREATE UNIQUE INDEX
"app_add_manytomany_field_b_many_to_many_b_id_a_id_3e15251d_uniq" ON
"app_add_manytomany_field_b_many_to_many" ("b_id", "a_id");
CREATE INDEX "app_add_manytomany_field_b_many_to_many_b_id_953b185b" ON
"app_add_manytomany_field_b_many_to_many" ("b_id");
CREATE INDEX "app_add_manytomany_field_b_many_to_many_a_id_4b44832a" ON
"app_add_manytomany_field_b_many_to_many" ("a_id");
COMMIT;
}}}
------
I could bisect it down to this commit
[https://code.djangoproject.com/changeset/2f73e5406d54cb8945e187eff302a3a3373350be
2f73e5406d54cb8945e187eff302a3a3373350be] (from #32502 and this
[https://github.com/django/django/pull/15175 PR]).
In the diff we see that the `# Special-case implicit M2M tables` comment
and its code were removed.
That's potentially a lead for a fix here I guess :)
(On a side note, this commit introduced another regression #33408. But
that's not related to the issue at hand)
Thank you!
--
Ticket URL: <https://code.djangoproject.com/ticket/34138>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: David Wobrock | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 4.1
Keywords: sqlite3 | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette):
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/34138#comment:1>
Django
Reporter: David Wobrock | Owner: Mariusz
| Felisiak
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Release blocker | Resolution:
Keywords: sqlite3 | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* owner: nobody => Mariusz Felisiak
* status: new => assigned
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/16254 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/34138#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: David Wobrock | Owner: Mariusz
| Felisiak
Component: Migrations | Version: 4.1
Severity: Release blocker | Resolution: fixed
Keywords: sqlite3 | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"7b0e9ea53ca99de2f485ec582f3a79be34b531d4" 7b0e9ea5]:
{{{
#!CommitTicketReference repository=""
revision="7b0e9ea53ca99de2f485ec582f3a79be34b531d4"
Fixed #34138 -- Avoided table rebuild when adding inline m2m fields on
SQLite.
Regression in 2f73e5406d54cb8945e187eff302a3a3373350be.
Thanks David Wobrock for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34138#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: David Wobrock | Owner: Mariusz
| Felisiak
Type: Bug | Status: closed
Component: Migrations | Version: 4.1
Severity: Release blocker | Resolution: fixed
Keywords: sqlite3 | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"84a2b2e7a7d8ac0dd210bcfa299fab8c4f240aae" 84a2b2e]:
{{{
#!CommitTicketReference repository=""
revision="84a2b2e7a7d8ac0dd210bcfa299fab8c4f240aae"
[4.1.x] Fixed #34138 -- Avoided table rebuild when adding inline m2m
fields on SQLite.
Regression in 2f73e5406d54cb8945e187eff302a3a3373350be.
Thanks David Wobrock for the report.
Backport of 7b0e9ea53ca99de2f485ec582f3a79be34b531d4 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34138#comment:4>