[Django] #26760: django_migrations table is never cleaned.
Django
-------------------------------------+-------------------------------------
Reporter: jarekwg | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Keywords: django_migrations
| squash migrations
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Django adds a `django_migrations` table to the database which list all of
the migrations that have been applied (and when).
With the introduction of `squashmigrations`, it is possible for this table
to contain a lot of old migrations that no longer exist. This can be
problematic if naming duplication occurs:
Example:
I have an app with:
{{{
my_app/migrations/
0001_initial.py
0002_blah.py
0003_blah.py
}}}
I squash and delete replaced migrations:
{{{
my_app/migrations/
0001_initial_squashed_0003_blah.py
}}}
I create a new migration and use poor naming:
{{{
my_app/migrations/
0001_initial_squashed_0003_blah.py
0002_blah.py
}}}
My new migration never runs because the `django_migrations` table thinks
it has already been applied.
I propose truncation of the `django_migrations` table so that it includes
only migrations that actually exist in the django project. This could be
done automatically (when executor runs, or inside the `migrate` mcommand).
Or have its own mcommand that requires it be run manually. I prefer the
automatic approach though.
Pros:
- Cleans up old data that just bloats the database.
- Protects users from the trap mentioned above where a new migration is
created with the same name as one that was applied in the past.
Cons:
- A loss of historical information.
Note:
Need to be careful with implementation to avoid a possible new trap if a
user squashes migrations and then proceeds to delete the replaced
migrations before running the squashed migrations on their database ->
django will think squashed migrations haven't been applied and will
attempt to reapply them. This can be remedied simply by not removing
migrations mentioned in `replaces` lists of other migrations from
`django_migrations` (ie. we'd consider replaced migrations as still
existing, even if their actual files have already been removed).
--
Ticket URL: <https://code.djangoproject.com/ticket/26760>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: jarekwg | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: => 0
* stage: Unreviewed => Accepted
* needs_tests: => 0
* needs_docs: => 0
Comment:
In #26429 we added a timestamp to merge migration names to reduce the
likelihood of collisions there. I acknowledge this could happen in other
situations though.
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:1>
Django
Reporter: jarekwg | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by shaib):
Note #25255 and #24900 - people sometimes still want to use the squashed
migrations (e.g. migrate back into the series that was squashed) in the
presence of the merged migration. I note that the suggestion is only to
remove from the database migrations whose files no longer exist. This
plays well with the suggestion to keep the migrations named in "replaces",
as we recommend that when the squashed migration files are removed, the
"replaces" clause is also removed from the migration.
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:2>
Django
Reporter: jarekwg | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by hellpain):
I think there is no need to delete records from db because they are almost
always generated automatically and have unique names.
I see one issue with automatic deleting of records: if I have django-apps
which are temporary disabled or are enabled and deploy in this moment,
data about their migrations will be lost.
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:3>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: dev@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:4>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Jacob Walls
* needs_better_patch: 0 => 1
* has_patch: 0 => 1
* status: new => assigned
Comment:
[https://github.com/django/django/pull/15240 PR]
Working on some test cleanup/isolation issues I just discovered.
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: django_migrations | Triage Stage: Accepted
squash migrations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
squash migrations | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: Jarek Glowacki | Owner: Jacob
Type: | Walls
Component: Migrations | Version: dev
Severity: Normal | Resolution: fixed
Keywords: django_migrations | Triage Stage: Ready for
squash migrations | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"2d8232fa716f5fe46eec9f35a0e90c2d0f126279" 2d8232fa]:
{{{
#!CommitTicketReference repository=""
revision="2d8232fa716f5fe46eec9f35a0e90c2d0f126279"
Fixed #26760 -- Added --prune option to migrate command.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26760#comment:14>